Pith. sign in

REVIEW 3 major objections 5 minor 61 references

Position Paper: Programming Language Techniques for Bridging LLM Code Generation Semantic Gaps

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM code generation can be made trustworthy only by integrating programming language techniques, this position paper argues.

desk verdict A readable, well-organized survey that re-frames existing PL-for-LLM work as an abstraction hierarchy, but the 'essential' claim it leads with is asserted, not argued, and one key citation is broken. read the letter →

arxiv 2507.09135 v1 pith:6YKENBEG submitted 2025-07-12 cs.SE

Yalong Du , Chaozheng Wang , Huaijin Wang This is my paper
classification cs.SE
keywords LLMcodegenerationsemanticgapsprogramminglanguagetechniquesformalverificationtypesystemsstructuredprogramrepresentationsabstractsyntaxtreesgenerate-verify
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This position paper argues that the semantic gaps in LLM-generated code—syntax errors, hallucinated APIs, subtle functional mistakes, and security weaknesses—cannot be closed by scaling alone. The authors claim that programming language techniques are essential: structured program representations, type systems, formal verification, and generate-verify testing should be integrated with neural generation. If right, future code-generation systems will be hybrid neuro-symbolic pipelines in which the LLM proposes code and PL machinery constrains, checks, and proves it. The paper organizes existing evidence for each layer and concludes with a call for deep integration rather than incremental prompting improvements.

What carries the argument

The mechanism that carries the argument is the layered integration of four PL artifact families: structured program representations (intermediate representations, abstract syntax trees, control-flow and data-flow graphs) that make structure explicit; type systems and constrained decoding that guide generation toward well-typed programs; formal verifiers and proof assistants that certify correctness properties; and generate-verify cycles combining testing, fuzzing, static analysis, and repair. Each layer is aimed at a specific rung of the error hierarchy, converting probabilistic pattern matching into constrained, checkable synthesis.

What would settle it

A controlled benchmark where a larger, unconstrained LLM paired with unit-test filtering matches or beats a smaller LLM wrapped in type constraints, formal verification, and static analysis on the same tasks would directly undercut the claim that PL integration is essential.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central claim is not a new experimental result but a diagnosis: the error patterns of LLM-generated code are hierarchical, ranging from syntax errors to runtime failures to functional correctness and code-quality hallucinations, and this hierarchy mirrors what programming language machinery already addresses. The paper therefore asserts that reliability requires converting code generation from unconstrained token prediction into a constrained process where structured representations supply syntax and dataflow, type systems guide decoding, formal verification establishes correctness, and generate-verify loops catch what slips through. The future of LLM-based code generation, in this view, lies in a paradigmatic shift toward deep integration of programming language theory with neural architectures.

Load-bearing premise

The argument assumes that current LLMs genuinely lack deep program semantics and that more data, better prompting, or test-based filtering cannot close the semantic gaps without formal PL machinery.

Editorial extensions

If this is right

  • Code-generation LLMs should be evaluated with PL-aware pipelines rather than by raw pass@k on unit tests alone.
  • Type-constrained decoding should become a default component for statically typed target languages, reducing compilation errors.
  • Formal verification becomes a practical complement to LLM generation by automating annotation and invariant synthesis, reducing manual specification effort.
  • Structured code representations should be native inputs or training objectives for code models, not post-hoc parsing targets.
  • Future architectures should incorporate dynamic execution feedback and graph-structured reasoning to handle runtime semantics.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Taken as a research agenda, the paper predicts that the value of PL scaffolding will grow for tasks requiring deep control-flow and data-flow reasoning; this could be tested by comparing constrained versus unconstrained generation across a difficulty spectrum.
  • If the thesis holds, the unit of evaluation shifts from the model to the pipeline: the LLM plus verifier, meaning model rankings may not transfer to deployed systems.
  • The emphasis on verifiability also raises a path for auditable correctness proofs around generated code, which could replace human review in regulated settings; the paper gestures at this but does not develop it.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper is a position paper about LLM code generation. It identifies syntactic, semantic, hallucination, reliability, and security failures; argues these stem from LLMs' token-stream processing and black-box nature; and surveys PL-based remedies such as IRs, AST/CFG/DFG representations, formal verification, type systems, testing/repair, static analysis, and internal-state analysis. It concludes that deep integration of PL theory with neural architectures is an essential paradigmatic shift. The paper contains no new experiments and builds its case exclusively from selected prior results.

Significance. The position is coherent and the survey is useful: it organizes a large recent literature, maps specific failure modes to concrete PL techniques, and flags dynamic semantics as an underexplored bottleneck. The manuscript credits real prior work throughout and does not invent evidence. Its main weakness is that the central claim is a strong modal one ('essential', 'paradigmatic shift') while the cited evidence supports only the weaker claim that PL techniques can help. As a roadmap the paper has value, but as an argument for necessity it is incomplete; the strongest counter-routes (scaling, prompting, test-based filtering) are not systematically weighed.

major comments (3)
  1. [Abstract; Section 6; §§3.2, 4.2, 5.1] The claim that PL integration is 'essential' and that the future is a 'paradigmatic shift' is the load-bearing assertion, but every quantitative result cited in support is an incremental improvement: AST-T5 gives 'substantial performance enhancements' (§3.2), type-constrained decoding 'significantly reduc[es] compilation errors' (§4.2), and CCTEST gives 'substantial accuracy improvements' (§5.1). Incremental improvements show usefulness, not necessity. The paper also contains an internal tension: §5.1 calls the 'generate-verify' cycle 'the most pragmatic and effective approach', which is a test-based route that does not require explicit PL machinery. To make the central claim defensible, the authors should either add a systematic comparison against scaling, prompting, and test-based filtering as alternative gap-closers, or weaken the claim to 'one important path'.
  2. [Section 2.2] The premise that LLMs 'fundamentally process code as sequential token streams, thereby disregarding its intrinsic structural properties' is asserted with citations [19] and [17] but not tested. Transformer attention over token streams can induce syntactic and semantic structure internally; indeed §5.3 cites [4] showing that internal states can signal code correctness. The paper needs either a direct experiment or a structured argument showing that the observed semantic gaps are causally due to token-stream processing and cannot be closed by additional data, better objectives, or scaffolding. As written, the premise is an unsupported dichotomy between token streams and explicit PL structure.
  3. [Section 3.2] The survey is one-sided. The paper cites LLMDFA and dataflow-sanitizing works [47,48] as evidence of 'substantial difficulties', but those works also report partially successful dataflow analysis by LLMs; the tension is not discussed. Similarly, the paper does not engage with the question raised by [39] about whether code LLMs already perform some static analysis, and it does not report any controlled comparison in which a non-PL baseline fails while the PL-augmented method succeeds. Because the central claim is necessity rather than helpfulness, ignoring partial counterevidence makes the argument incomplete.
minor comments (5)
  1. [Reference [19]] Reference [19] in Section 2.2 is formatted as 'TRASTIVE LEARNING. Nova: ...' with no author names; complete bibliographic information is needed.
  2. [Section 2.3] The statistic 'approximately 40% of code containing potential vulnerabilities' is attributed to [58], but [58] is a broad survey; please cite the primary vulnerability study or qualify the provenance.
  3. [Section 4.1] The tool named 'Marmaragan' appears to be misspelled; please verify the name against [7].
  4. [Section 3.2] The sentence beginning 'The recognition that raw token processing proves inadequate... mandates explicit encoding' reads as a conclusion but has no citation or argument immediately following; support it or soften it.
  5. [Section 6] The conclusion's emphasis on interpretability goes beyond what the body establishes: Section 5.3 is short and speculative, so the conclusion should mark interpretability as a forward-looking goal rather than an established benefit.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's central claim is a position argument supported by external empirical evidence, not a derivation that presupposes its conclusion.

full rationale

This is a position paper, not a derivation chain: it contains no equations, fitted parameters, or predictions that reduce to their inputs by construction. The core claim that PL techniques are 'essential' is a normative/necessity argument supported by a survey of published empirical results, not a construction that builds the conclusion into its premises. Section 2.2's premise that LLMs process code as linear token streams is an empirical assertion cited to external works ([17], [19], [36]); even if that premise is contestable or weakly supported, it is not equivalent to the paper's own conclusion and does not make the argument circular. The many self-citations (e.g., CCTEST in §5.1, IR embeddings in §3.1, watermarking in §2.3) are to externally published, independently reviewed empirical studies; none is used as a uniqueness theorem or as a substitute for an argument that assumes the conclusion. The paper explicitly frames itself as a position paper, and its strongest conclusion in Section 6 is a proposed research agenda, not a derived theorem. No circular step can be quoted because no load-bearing step equates its output to its input by definition or by self-referential fit.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The thesis rests on three domain assumptions about LLM limitations, PL technique efficacy, and the severity of semantic gaps. There are no free parameters or invented entities.

assumptions (3)
  • domain assumption LLMs process code as linear token streams and lack deep program semantics understanding.
    Section 2.2 introduces this as the reason PL-based structure is needed; it is supported by citations but not demonstrated in this paper.
  • domain assumption Integrating PL representations and verification into LLM workflows can substantially improve reliability without prohibitive cost.
    This is the paper's central thesis. Only supportive examples are cited; no comparative cost or failure analysis is provided.
  • domain assumption Semantic gaps in LLM-generated code, including syntax errors and security vulnerabilities, are prevalent and important.
    Sections 2.1 and 2.3 rely on external empirical studies, including an attributed statistic that roughly 40% of Copilot-generated code contains vulnerabilities.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Position Paper: Programming Language Techniques for Bridging LLM Code Generation Semantic Gaps." pith.science (2026). https://pith.science/paper/6YKENBEG

@misc{pith2026250709135,
  author       = {Pith},
  title        = {Pith review of: Position Paper: Programming Language Techniques for Bridging LLM Code Generation Semantic Gaps},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6YKENBEG}},
  note         = {Machine review of arXiv:2507.09135}
}
read the original abstract

Large Language Models have demonstrated remarkable capabilities in automated code generation, yet their statistical nature and black-box characteristics create significant semantic gaps manifested through syntax errors, semantic hallucinations, and reliability concerns. This position paper argues that principled integration of Programming Language (PL) techniques is essential for bridging these gaps. Through structured program representations, formal correctness guarantees, and robust verification mechanisms, PL techniques can elevate LLM-generated code from statistical pattern matching to truly reliable and trustworthy levels. This integration is crucial for developing systems that generate code that is not only functionally correct but also interpretable, verifiable, and ultimately trustworthy.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

61 extracted references · 37 canonical work pages

  1. [19]

    Nova: Generative language models for assem- bly code with hierarchical attention and con-trastive learning

    TRASTIVE LEARNING. Nova: Generative language models for assem- bly code with hierarchical attention and con-trastive learning

    work page

  2. [17]

    Can large language models understand intermediate representations? arXiv preprint arXiv:2502.06854, 2025

    Hailong Jiang, Jianfeng Zhu, Yao Wan, Bo Fang, Hongyu Zhang, Ruom- ing Jin, and Qiang Guan. Can large language models understand intermediate representations? arXiv preprint arXiv:2502.06854, 2025

    arXiv 2025

  3. [4]

    Correctness assessment of code generated by large language models using internal representations

    Tuan-Dung Bui, Thanh Trong Vu, Thu-Trang Nguyen, Son Nguyen, and Hieu Dinh Vo. Correctness assessment of code generated by large language models using internal representations. arXiv preprint arXiv:2501.12934, 2025

    arXiv 2025

  4. [39]

    Do code llms do static analysis? arXiv preprint arXiv:2505.12118, 2025

    Chia-Yi Su and Collin McMillan. Do code llms do static analysis? arXiv preprint arXiv:2505.12118, 2025

    arXiv 2025

  5. [1]

    Automatic semantic augmentation of language model prompts (for code summarization)

    Toufique Ahmed, Kunal Suresh Pai, Premkumar Devanbu, and Earl Barr. Automatic semantic augmentation of language model prompts (for code summarization). In Proceedings of the IEEE/ACM 46th inter- national conference on software engineering , pages 1–13, 2024. Position Paper: Programming Language Techniques for Bridging LLM Code Generation Semantic Gaps

    work page 2024

  6. [2]

    Enhancing llm code generation: A systematic evaluation of multi-agent collabo- ration and runtime debugging for improved accuracy, reliability, and latency

    Nazmus Ashrafi, Salah Bouktif, and Mohammed Mediani. Enhancing llm code generation: A systematic evaluation of multi-agent collabo- ration and runtime debugging for improved accuracy, reliability, and latency. arXiv preprint arXiv:2505.02133, 2025

    arXiv 2025

  7. [3]

    Statically contextualizing large language models with typed holes

    Andrew Blinn, Xiang Li, June Hyung Kim, and Cyrus Omar. Statically contextualizing large language models with typed holes. Proceedings of the ACM on Programming Languages , 8(OOPSLA2):468–498, 2024

    work page 2024

  8. [5]

    Empirical evaluation of generalizable automated program repair with large language models

    Viola Campos, Ridwan Shariffdeen, Adrian Ulges, and Yannic Noller. Empirical evaluation of generalizable automated program repair with large language models. arXiv preprint arXiv:2506.03283, 2025

    arXiv 2025

Show all 61 references
  1. [6]

    Polyver: A compositional approach for polyglot system modeling and verification.arXiv preprint arXiv:2503.03207, 2025

    Pei-Wei Chen, Shaokai Lin, Adwait Godbole, Ramneet Singh, Elizabeth Polgreen, Edward A Lee, and Sanjit A Seshia. Polyver: A compositional approach for polyglot system modeling and verification.arXiv preprint arXiv:2503.03207, 2025

    2025

  2. [7]

    Verifying llm-generated code in the context of software verification with ada/spark

    Marcos Cramer and Lucian McIntyre. Verifying llm-generated code in the context of software verification with ada/spark. arXiv preprint arXiv:2502.07728, 2025

    2025 arXiv

  3. [8]

    Search- based llms for code optimization

    Shuzheng Gao, Cuiyun Gao, Wenchao Gu, and Michael Lyu. Search- based llms for code optimization. In 2025 IEEE/ACM 47th International Conference on Software Engineering (ICSE) , pages 254–266. IEEE Com- puter Society, 2024

    2025

  4. [9]

    Shuzheng Gao, Cuiyun Gao, Yulan He, Jichuan Zeng, Lunyiu Nie, Xin Xia, and Michael R. Lyu. Code structure-guided transformer for source code summarization. ACM Trans. Softw. Eng. Methodol. , 32(1):23:1–23:32, 2023

    2023

  5. [10]

    Two sides of the same coin: Exploiting the impact of identifiers in neural code comprehension

    Shuzheng Gao, Cuiyun Gao, Chaozheng Wang, Jun Sun, David Lo, and Yue Yu. Two sides of the same coin: Exploiting the impact of identifiers in neural code comprehension. In 2023 IEEE/ACM 45th International Conference on Software Engineering (ICSE) , pages 1933–1945. IEEE, 2023

    2023

  6. [11]

    Shuzheng Gao, Xin-Cheng Wen, Cuiyun Gao, Wenxuan Wang, Hongyu Zhang, and Michael R Lyu. What makes good in-context demonstrations for code intelligence tasks with llms? In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE), pages 761–773. IEEE, 2023

    2023

  7. [12]

    Ast-t5: Structure- aware pretraining for code generation and understanding

    Linyuan Gong, Mostafa Elhoushi, and Alvin Cheung. Ast-t5: Structure- aware pretraining for code generation and understanding. arXiv preprint arXiv:2401.03003, 2024

    2024 arXiv

  8. [13]

    Large language models for code genera- tion: A comprehensive survey of challenges, techniques, evaluation, and applications

    Nam Huynh and Beiyu Lin. Large language models for code genera- tion: A comprehensive survey of challenges, techniques, evaluation, and applications. arXiv preprint arXiv:2503.01245, 2025

    2025 arXiv

  9. [14]

    Large language models (llms) for source code analysis: applications, models and datasets

    Hamed Jelodar, Mohammad Meymani, and Roozbeh Razavi-Far. Large language models (llms) for source code analysis: applications, models and datasets. arXiv preprint arXiv:2503.17502, 2025

    2025 arXiv

  10. [15]

    Causality-aided evaluation and explanation of large language model- based code generation

    Zhenlan Ji, Pingchuan Ma, Zongjie Li, Zhaoyu Wang, and Shuai Wang. Causality-aided evaluation and explanation of large language model- based code generation. In Proceedings of the 34th ACM SIGSOFT Inter- national Symposium on Software Testing and Analysis , 2025

    2025

  11. [16]

    Testing and understanding erroneous planning in llm agents through synthesized user inputs

    Zhenlan Ji, Daoyuan Wu, Pingchuan Ma, Zongjie Li, and Shuai Wang. Testing and understanding erroneous planning in llm agents through synthesized user inputs. arXiv preprint arXiv:2404.17833, 2024

    2024 arXiv

  12. [18]

    Codecrash: Stress testing llm reasoning under structural and semantic perturbations

    Man Ho Lam, Chaozheng Wang, Jen-tse Huang, and Michael R Lyu. Codecrash: Stress testing llm reasoning under structural and semantic perturbations. arXiv preprint arXiv:2504.14119, 2025

    2025

  13. [20]

    Hallucination by code generation llms: Taxonomy, benchmarks, mitigation, and challenges

    Yunseo Lee, John Youngeun Song, Dongsun Kim, Jindae Kim, Mijung Kim, and Jaechang Nam. Hallucination by code generation llms: Taxonomy, benchmarks, mitigation, and challenges. arXiv preprint arXiv:2504.20799, 2025

    2025 arXiv

  14. [21]

    The hitchhiker’s guide to program analysis, part ii: Deep thoughts by llms

    Haonan Li, Hang Zhang, Kexin Pei, and Zhiyun Qian. The hitchhiker’s guide to program analysis, part ii: Deep thoughts by llms. arXiv preprint arXiv:2504.11711, 2025

    2025 arXiv

  15. [22]

    Large language model powered symbolic execution

    Yihe Li, Ruijie Meng, and Gregory J Duck. Large language model powered symbolic execution. arXiv preprint arXiv:2505.13452, 2025

    2025

  16. [23]

    Unleashing the power of compiler intermedi- ate representation to enhance neural program embeddings

    Zongjie Li, Pingchuan Ma, Huaijin Wang, Shuai Wang, Qiyi Tang, Sen Nie, and Shi Wu. Unleashing the power of compiler intermedi- ate representation to enhance neural program embeddings. In 44th IEEE/ACM 44th International Conference on Software Engineering, ICSE 2022, Pittsburg...

    2022

  17. [24]

    An empirical study on large language models in accuracy and robustness under chinese industrial scenarios

    Zongjie Li, Wenying Qiu, Pingchuan Ma, Yichen Li, You Li, Sijia He, Baozheng Jiang, Shuai Wang, and Weixi Gu. An empirical study on large language models in accuracy and robustness under chinese industrial scenarios. arXiv preprint arXiv:2402.01723, 2024

    2024 arXiv

  18. [25]

    On the accuracy and robustness of large language models in chinese industrial scenarios

    Zongjie Li, Wenying Qiu, Pingchuan Ma, Yichen Li, You Li, Sijia He, Baozheng Jiang, Shuai Wang, and Weixi Gu. On the accuracy and robustness of large language models in chinese industrial scenarios. In 2024 23rd ACM/IEEE International Conference on Information Processing in Se...

    2024

  19. [26]

    CCTEST: testing and repairing code completion systems

    Zongjie Li, Chaozheng Wang, Zhibo Liu, Haoxuan Wang, Dong Chen, Shuai Wang, and Cuiyun Gao. CCTEST: testing and repairing code completion systems. In 45th IEEE/ACM International Conference on Software Engineering, ICSE 2023, Melbourne, Australia, May 14-20, 2023 , pages 1238–1...

    2023

  20. [27]

    On the feasibility of specialized ability stealing for large language code models

    Zongjie Li, Chaozheng Wang, Pingchuan Ma, Chaowei Liu, Shuai Wang, Daoyuan Wu, and Cuiyun Gao. On the feasibility of specialized ability stealing for large language code models. 2023

    2023

  21. [28]

    Split and merge: Aligning position biases in LLM-based evaluators

    Zongjie Li, Chaozheng Wang, Pingchuan Ma, Daoyuan Wu, Shuai Wang, Cuiyun Gao, and Yang Liu. Split and merge: Aligning position biases in LLM-based evaluators. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen, editors, Proceedings of the 2024 Conference on Empirical Methods...

    2024

  22. [29]

    Protect- ing intellectual property of large language model-based code genera- tion apis via watermarks

    Zongjie Li, Chaozheng Wang, Shuai Wang, and Gao Cuiyun. Protect- ing intellectual property of large language model-based code genera- tion apis via watermarks. In Proceedings of the 2023 ACM SIGSAC Con- ference on Computer and Communications Security, CCS 2023, Copen- hagen, D...

    2023

  23. [30]

    Reasoning as a resource: Optimizing fast and slow thinking in code generation models

    Zongjie Li and Shuai Wang. Reasoning as a resource: Optimizing fast and slow thinking in code generation models. arXiv preprint arXiv:2506.09396, 2025

    2025 arXiv

  24. [31]

    Api-guided dataset synthesis to finetune large code models

    Zongjie Li, Daoyuan Wu, Shuai Wang, and Zhendong Su. Api-guided dataset synthesis to finetune large code models. Proceedings of the ACM on Programming Languages, 9(OOPSLA1):786–815, 2025

    2025

  25. [32]

    Differentiation-based extraction of proprietary data from fine-tuned llms

    Zongjie Li, Daoyuan Wu, Shuai Wang, and Su Zhendong. Differentiation-based extraction of proprietary data from fine-tuned llms. In Proceedings of the 2025 ACM SIGSAC Conference on Computer and Communications Security, CCS 2025, Taipei, Taiwan, October 13-17, 2025, 2025

    2025

  26. [33]

    Large language models can be guided to evade ai-generated text de- tection

    Ning Lu, Shengcai Liu, Rui He, Yew-Soon Ong, Qi Wang, and Ke Tang. Large language models can be guided to evade ai-generated text de- tection. Trans. Mach. Learn. Res. , 2024

    2024

  27. [34]

    Safe delta: Consistently preserving safety when fine-tuning LLMs on diverse datasets

    Ning Lu, Shengcai Liu, Jiahao Wu, Weiyu Chen, Zhirui Zhang, Yew- Soon Ong, Qi Wang, and Ke Tang. Safe delta: Consistently preserving safety when fine-tuning LLMs on diverse datasets. In Forty-second International Conference on Machine Learning , 2025

    2025

  28. [35]

    Less is More: Understanding Word-level Textual Adversarial Attack via n-gram Frequency Descend

    Ning Lu, Shengcai Liu, Zhirui Zhang, Qi Wang, Haifeng Liu, and Ke Tang. Less is More: Understanding Word-level Textual Adversarial Attack via n-gram Frequency Descend . In 2024 IEEE Conference on Artificial Intelligence, pages 823–830, 2024. Yalong Du, Chaozheng Wang, and Huaijin Wang

    2024

  29. [36]

    Lms: Understanding code syntax and semantics for code analysis.arXiv preprint arXiv:2305.12138, 2023

    Wei Ma, Shangqing Liu, Zhihao Lin, Wenhan Wang, Qiang Hu, Ye Liu, Cen Zhang, Liming Nie, Li Li, and Yang Liu. Lms: Understanding code syntax and semantics for code analysis.arXiv preprint arXiv:2305.12138, 2023

    2023 arXiv

  30. [37]

    Type-constrained code generation with language models

    Niels Mündler, Jingxuan He, Hao Wang, Koushik Sen, Dawn Song, and Martin Vechev. Type-constrained code generation with language models. arXiv preprint arXiv:2504.09246, 2025

    2025 arXiv

  31. [38]

    Static inference meets deep learning: a hybrid type inference approach for python

    Yun Peng, Cuiyun Gao, Zongjie Li, Bowei Gao, David Lo, Qirun Zhang, and Michael Lyu. Static inference meets deep learning: a hybrid type inference approach for python. In Proceedings of the 44th International Conference on Software Engineering , pages 2019–2030, 2022

    2019

  32. [40]

    Classinvgen: Class invariant synthesis using large language models

    Chuyue Sun, Viraj Agashe, Saikat Chakraborty, Jubi Taneja, Clark Barrett, David Dill, Xiaokang Qiu, and Shuvendu K Lahiri. Classinvgen: Class invariant synthesis using large language models. arXiv preprint arXiv:2502.18917, 2025

    2025 arXiv

  33. [41]

    Beyond peft: Layer-wise optimization for more effective and efficient large code model tuning

    Chaozheng Wang, Jia Feng, Shuzheng Gao, Cuiyun Gao, Zongjie Li, Ting Peng, Hailiang Huang, Yuetang Deng, and Michael Lyu. Beyond peft: Layer-wise optimization for more effective and efficient large code model tuning. Proceedings of the ACM on Software Engineering , 2(FSE):1567...

    2025

  34. [42]

    A systematic evaluation of large code models in api suggestion: When, which, and how

    Chaozheng Wang, Shuzheng Gao, Cuiyun Gao, Wenxuan Wang, Chun Yong Chong, Shan Gao, and Michael R Lyu. A systematic evaluation of large code models in api suggestion: When, which, and how. In Proceedings of the 39th IEEE/ACM International Conference on Automated Software Engine...

    2024

  35. [43]

    Reef: A framework for collecting real-world vulnerabilities and fixes

    Chaozheng Wang, Zongjie Li, Yun Pena, Shuzheng Gao, Sirong Chen, Shuai Wang, Cuiyun Gao, and Michael R Lyu. Reef: A framework for collecting real-world vulnerabilities and fixes. In 2023 38th IEEE/ACM International Conference on Automated Software Engineering (ASE) , pages 195...

    2023

  36. [44]

    Enriching query semantics for code search with reinforcement learning

    Chaozheng Wang, Zhenhao Nong, Cuiyun Gao, Zongjie Li, Jichuan Zeng, Zhenchang Xing, and Yang Liu. Enriching query semantics for code search with reinforcement learning. Neural Networks, 145:22–32, 2022

    2022

  37. [45]

    Prompt tuning in code intelligence: An experimental evaluation

    Chaozheng Wang, Yuanhang Yang, Cuiyun Gao, Yun Peng, Hongyu Zhang, and Michael R Lyu. Prompt tuning in code intelligence: An experimental evaluation. IEEE Transactions on Software Engineering , 49(11):4869–4885, 2023

    2023

  38. [46]

    Rag or fine- tuning? a comparative study on lcms-based code completion in indus- try

    Chaozheng Wang, Zezhou Yang, Shuzheng Gao, Cuiyun Gao, Ting Peng, Hailiang Huang, Yuetang Deng, and Michael Lyu. Rag or fine- tuning? a comparative study on lcms-based code completion in indus- try. arXiv preprint arXiv:2505.15179, 2025

    2025 arXiv

  39. [47]

    Llmdfa: Analyzing dataflow in code with large language models

    Chengpeng Wang, Wuqi Zhang, Zian Su, Xiangzhe Xu, Xiaoheng Xie, and Xiangyu Zhang. Llmdfa: Analyzing dataflow in code with large language models. Advances in Neural Information Processing Systems , 37:131545–131574, 2024

    2024

  40. [48]

    Sanitizing large language models in bug detection with data- flow

    Chengpeng Wang, Wuqi Zhang, Zian Su, Xiangzhe Xu, and Xiangyu Zhang. Sanitizing large language models in bug detection with data- flow. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 3790–3805, 2024

    2024

  41. [49]

    sem2vec: Semantics-aware assembly tracelet embedding

    Huaijin Wang, Pingchuan Ma, Shuai Wang, Qiyi Tang, Sen Nie, and Shi Wu. sem2vec: Semantics-aware assembly tracelet embedding. ACM Trans. Softw. Eng. Methodol., 32(4):1–34, 2023

    2023

  42. [50]

    Navrepair: Node-type aware c/c++ code vulnerability repair

    Ruoke Wang, Zongjie Li, Chaozheng Wang, Yang Xiao, and Cuiyun Gao. Navrepair: Node-type aware c/c++ code vulnerability repair. arXiv preprint arXiv:2405.04994, 2024

    2024

  43. [51]

    Towards understanding the characteristics of code generation errors made by large language models

    Zhijie Wang, Zijie Zhou, Yuheng Huang Da Song, Shengmai Chen, Lei Ma, and Tianyi Zhang. Towards understanding the characteristics of code generation errors made by large language models. Preprint, 2025

    2025

  44. [52]

    Zhang, and Qing Liao

    Xin-Cheng Wen, Yupan Chen, Cuiyun Gao, Hongyu Zhang, Jie M. Zhang, and Qing Liao. Vulnerability detection with graph simplifica- tion and enhanced graph representation learning. In 45th IEEE/ACM International Conference on Software Engineering, ICSE 2023, Melbourne, Australia,...

    2023

  45. [53]

    Vuleval: Towards repository-level evaluation of software vulnerability detection

    Xin-Cheng Wen, Xinchen Wang, Yujia Chen, Ruida Hu, David Lo, and Cuiyun Gao. Vuleval: Towards repository-level evaluation of software vulnerability detection. CoRR, abs/2404.15596, 2024

    2024 arXiv

  46. [54]

    Refining decompiled c code with large language models

    Wai Kin Wong, Huaijin Wang, Zongjie Li, Zhibo Liu, Shuai Wang, Qiyi Tang, Sen Nie, and Shi Wu. Refining decompiled c code with large language models. arXiv preprint arXiv:2310.06530, 2023

    2023 arXiv

  47. [55]

    BinAug: Enhancing binary similarity analysis with low-cost input repairing

    Wai Kin Wong, Huaijin Wang, Zongjie Li, and Shuai Wang. BinAug: Enhancing binary similarity analysis with low-cost input repairing. In Proceedings of the 46th IEEE/ACM International Conference on Software Engineering (ICSE), pages 1–13, 2024

    2024

  48. [56]

    Decllm: Llm-augmented recompilable decompilation for enabling programmatic use of decom- piled code

    Wai Kin Wong, Daoyuan Wu, Huaijin Wang, Zongjie Li, Zhibo Liu, Shuai Wang, Qiyi Tang, Sen Nie, and Shi Wu. Decllm: Llm-augmented recompilable decompilation for enabling programmatic use of decom- piled code. In Proceedings of the 34th ACM SIGSOFT International Symposium on Sof...

    2025

  49. [57]

    Formal mathematical reasoning: A new frontier in ai

    Kaiyu Yang, Gabriel Poesia, Jingxuan He, Wenda Li, Kristin Lauter, Swarat Chaudhuri, and Dawn Song. Formal mathematical reasoning: A new frontier in ai. arXiv preprint arXiv:2412.16075, 2024

    2024 arXiv

  50. [58]

    Robustness, security, privacy, explainability, efficiency, and usability of large language models for code

    Zhou Yang, Zhensu Sun, Terry Zhuo Yue, Premkumar Devanbu, and David Lo. Robustness, security, privacy, explainability, efficiency, and usability of large language models for code. arXiv preprint arXiv:2403.07506, 2024

    2024 arXiv

  51. [59]

    Order matters: Semantic-aware neural networks for binary code similarity detection

    Zeping Yu, Rui Cao, Qiyi Tang, Sen Nie, Junzhou Huang, and Shi Wu. Order matters: Semantic-aware neural networks for binary code similarity detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 1145–1152, 2020

    2020

  52. [60]

    Low-cost and comprehensive non-textual input fuzzing with llm-synthesized input generators

    Kunpeng Zhang, Zongjie Li, Daoyuan Wu, Shuai Wang, and Xin Xia. Low-cost and comprehensive non-textual input fuzzing with llm-synthesized input generators. arXiv preprint arXiv:2501.19282 , 2025

    2025 arXiv

  53. [61]

    An approach for api synthesis using large language models.arXiv preprint arXiv:2502.15246, 2025

    Hua Zhong, Shan Jiang, and Sarfraz Khurshid. An approach for api synthesis using large language models.arXiv preprint arXiv:2502.15246, 2025

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.