Pith. sign in

REVIEW 5 major objections 5 minor 126 references

Code generation improves when a model tracks which code entities depend on which, and accepts a candidate only after it agrees with both semantic and structural views.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 20:10 UTC pith:2P6SSOOC

load-bearing objection Plausible framework, but the central dependency-consistency mechanism is undefined as written and the ablation doesn't isolate it; worth review but needing major revision. the 5 major comments →

arxiv 2607.16692 v2 pith:2P6SSOOC submitted 2026-07-18 cs.SE cs.CL

Dependency-Guided Code Generation: Structured Matrix Decomposition and Consistency-Guided Refinement

Mingqiao Mo , Yangchen Zeng , Zikai Xiao , Xin Xiao , Wenhua Nie , Zhaolu Kang , Guangyuan Dong , Kai Shu
show 2 more authors
Hao Zhang Xiaodong Fan
This is my paper
classification cs.SE cs.CL
keywords dependency-aware code generationcode entity dependency graphquantized matrix decompositionlow-rank factorizationalternating optimizationconsistency-guided refinementsparse triplet storagestructural code retrieval
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Modern code generators produce locally fluent code but routinely miss the imports, calls, inheritance, and cross-file links that make code usable in a real system. The paper argues that this gap can be closed by making dependency structure explicit: represent a codebase as a graph, decompose the graph's adjacency matrix into a quantized matrix holding strong discrete dependencies and a sparse low-rank factor holding weaker implicit couplings, and then require every generated candidate to pass a consistency check between semantic similarity and structural dependency retrieval before it is accepted. If a candidate fails, the dependency rows are fed back into the prompt and the model regenerates. The paper reports that this dependency-guided refinement consistently beats the base generation model and prior structure-aware baselines on code translation, text-to-code, and program-synthesis benchmarks, with the largest gains on exact-match and execution-based metrics. A sympathetic reader would care because it treats code generation as a structural integration problem rather than a token-fluency problem, which is the failure mode that matters when models generate code for real codebases.

Core claim

The central discovery is that a single codebase graph can be split into two complementary dependency signals, and that this split pays off at generation time. Strong, explicit relations — function calls, inheritance, imports — are encoded in a quantized matrix Q whose entries are snapped to a discrete codebook; weak, implicit, higher-order interactions are absorbed by a low-rank product AB^T. Q and (A,B) are learned by alternating minimization, and the strong entries are stored as sparse triplets to keep storage linear in the number of meaningful dependencies. During generation, a candidate snippet is embedded, semantically similar entities and structurally related entities are retrieved in

What carries the argument

The load-bearing mechanism is the hybrid decomposition W ≈ Q + AB^T combined with the consistency-guided refinement loop. W is the adjacency matrix of the code-entity graph; Q is a quantized matrix with entries drawn from a discrete codebook, capturing strong, sparse, interpretable dependencies such as calls and imports; AB^T is a low-rank factorization that represents weak, implicit, higher-order couplings. An alternating scheme projects the residual W − AB^T onto the codebook to update Q, then fits A and B to W − Q, with Frobenius regularization. The learned structure is used at generation time through a dual-channel retrieval: semantic top-k by embedding cosine similarity and structural t

Load-bearing premise

The load-bearing premise is that every generated candidate snippet can be associated with some node of the dependency graph so that its structural row Q_{c,:} exists; the paper does not specify how that association is made from raw code, and if the mapping is ill-posed the structural retrieval and consistency check have no well-defined input.

What would settle it

A direct check: run the full framework on a program-synthesis benchmark with the structural retrieval channel replaced by retrieval over random rows of Q drawn from the same distribution, keeping all other components and the threshold fixed. If test-case accuracy does not drop below the reported 'w/o Constraint' ablation, the gains are not attributable to dependency structure. A complementary check is to take generated snippets that mention entities absent from the dependency graph and count how often Q_{c,:} is actually computable; if such cases are frequent, the mapping assumption fails on t

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • If the claim is right, code generators can gain repository-level structural awareness without retraining the base model: the dependency matrix and the check-refine loop are added at inference time.
  • The sparse triplet representation makes the approach scalable to large codebases, since storage and retrieval depend on the number of strong dependencies, not on all pairwise entries.
  • Separating strong from weak dependencies gives an interpretable account of why a candidate is rejected: its semantic neighbors and structural neighbors disagree, and the structural list provides a concrete repair signal.
  • The decomposition itself becomes a reusable artifact: the quantized matrix can serve as a compact dependency feature for other tasks such as code retrieval, summarization, or repository navigation.
  • Gains concentrated in exact-match and execution-based metrics imply the mechanism fixes integration and correctness errors, not just surface phrasing.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the paper never specifies how a generated snippet is mapped to a node in the dependency graph to obtain Q_{c,:}; a natural and testable extension is to ground candidates to nodes through resolved symbols and imports, and to report how often this mapping is ambiguous or fails.
  • Editorial inference: because the refinement loop only reprompts the same generator, it may inherit the generator's blind spots; coupling the dependency score to the training objective or to constrained decoding is a natural next experiment.
  • Editorial inference: the decomposition is generic in the sense that any domain with a definable relation matrix — API schemas, data contracts, document outlines — could use the same accept-or-refine pattern; the paper's own limitation note about dynamic contexts suggests updating W with runtime or version-history signals would be a stronger stress test.
  • Editorial inference: a fixed acceptance threshold assumes one consistency tolerance fits all tasks; per-task calibration or a learned threshold is a cheap experiment that would show whether the reported robustness generalizes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes a dependency-aware code generation framework. It represents a codebase as a directed graph of code entities, encodes interactions in an adjacency matrix W, and decomposes W as Q + AB^T, where Q is a quantized matrix for strong explicit dependencies and AB^T is a low-rank factor for weak implicit dependencies. The decomposition is learned by alternating optimization. During generation, the framework retrieves semantically similar entities and structurally related entities, checks their consistency via a Jaccard similarity, and iteratively refines the prompt until the threshold is met. Experiments on CodeXGLUE translation, CONCODE, and APPS report improvements over CodeT5 and other baselines, with ablations and sensitivity analyses. The central claim is that modeling and enforcing dependency structure improves semantic alignment and structural fidelity of generated code.

Significance. The topic is relevant and the high-level idea—separating strong, discrete dependencies from weak, latent interactions and using them to guide generation—is potentially interesting. The paper evaluates on standard external benchmarks (BLEU, CodeBLEU, test-case accuracy), so the headline numbers are not circular. The sparse triplet storage idea could also be useful if properly defined. However, the manuscript currently leaves the central dependency-modeling mechanism under-specified: W is never constructed, the mapping from a generated snippet to a graph vertex is not defined, and the ablations do not isolate the dependency component from plain semantic retrieval. As a result, the main empirical claim cannot yet be validated. These are load-bearing issues rather than presentation issues.

major comments (5)
  1. [§3.2, Eq. (1)] The adjacency matrix W, which is the input to every subsequent equation, is never constructed. The paper states that W_ij measures interaction strength but does not specify how functions, classes, imports, inheritance, or calls are extracted from source code, how the entity set V is defined for the benchmarks, or what static-analysis procedure generates the graph. For CodeXGLUE translation and CONCODE, which are not repository-level tasks, it is especially unclear what the codebase graph contains. Without this construction, Eqs. (1)–(7) cannot be reproduced and the empirical results cannot be checked.
  2. [§3.4, Eq. (9)] Eq. (9) defines S_Q = Top-k(Q_{ĉ,:}), but Q is indexed by codebase entities V (functions, classes, files), while ĉ is a generated text snippet. No mapping from ĉ to a vertex of G is provided, and no extension of W to generated text is defined. Consequently the Jaccard consistency check in Eq. (10), the accept/refine criterion in Section 3.4, and the dependency-consistency analysis in Figure 2a are undefined. The central refinement mechanism cannot be executed or validated as written.
  3. [§3.5, Eq. (11)] The claimed sparsity of T(Q) is internally inconsistent. With the implementation codebook q_1,...,q_8 = 1,2,...,8 and Q_ij assigned to one of these levels for every entry, every Q_ij satisfies Q_ij ≥ 1 = q_1. Thus if the threshold in Eq. (11) is q_1, T(Q) contains all n^2 entries; if a higher level is intended, the threshold index k is never defined and no zero/null level is introduced in Eq. (3). As written, the O(|T(Q)|) storage claim and the scalability contribution are not established.
  4. [§4.4, Table 4] The ablations remove Q and the consistency constraint but do not remove or randomize the semantic retrieval channel of Eq. (8). The full model is CodeT5 augmented with both semantic retrieval and structural retrieval, so the reported +1.9 BLEU / +3.4 xMatch over CodeT5 could be attributable to retrieval-augmented CodeT5 alone rather than to the dependency decomposition or consistency refinement. A control with only semantic retrieval (and no dependency component) is needed to support the central claim. In addition, no variance or significance tests are reported; the deltas are small relative to typical noise on BLEU/CodeBLEU.
  5. [§4.1] The objective in Eq. (2) contains regularization coefficients λ_A and λ_B, but their values or selection procedure are never reported. The number of refinement iterations, the decoding/retrieval budget, and the exact procedure used to set τ = 0.8 are also unspecified. This, together with the missing W construction and snippet-to-vertex mapping, makes the experiments impossible to reproduce from the text.
minor comments (5)
  1. [Notation, Eq. (11)] The symbol k is used both for the low-rank dimension (k=8) and for the codebook index in Eq. (11), which is confusing. Please rename one of them and define q_k explicitly.
  2. [Figure 2b] The sensitivity analysis around τ is described only qualitatively. Report the actual BLEU/CodeBLEU values and, ideally, variance over runs; as presented, it is not possible to assess how stable performance is across thresholds.
  3. [Table 3] The column header 'Test Case Avg. Strict Acc.' is ambiguous: there are no average columns in the table body. Please clarify whether the reported numbers are per-split averages or standard APPS difficulty-level results.
  4. [§4.4, Table 4] The 'w/o Constraint' variant (BLEU 83.2) is below the CodeT5 backbone (BLEU 84.2) on the same task. Please explain why removing the constraint degrades performance below the baseline, or state clearly which other components remain in this variant.
  5. [§3.2 and Limitations] The Limitations section acknowledges that static dependency extraction may be insufficient, but it does not address the more basic concern that the static extraction procedure itself is never described. Please specify the extraction method or point to released code/data.

Circularity Check

0 steps flagged

No significant circularity: the headline benchmark gains are measured against external metrics, and the main formal gaps (undefined snippet-to-entity mapping for Eq. 9, unspecified construction of W) are correctness risks, not circular reductions.

full rationale

The paper's central derivation is the decomposition W ≈ Q + AB^T (Eq. 1), solved by alternating updates (Eqs. 4–7). This is a parameterization of W, not a prediction derived from it: Q, A, and B are fit to W, and the headline claims are evaluated against external metrics (CodeXGLUE BLEU/xMatch/CodeBLEU, CONCODE, APPS test-case accuracy). No fitted coefficient is renamed as a held-out prediction. Self-citations in the introduction, related work, and limitations are contextual and none supplies a load-bearing premise; in particular, no uniqueness theorem or prior-work ansatz is imported to force the decomposition. The strongest concerns are formal incompleteness: Eq. (9) writes S_Q = Top-k(Q_{\hat c,:}) with Q indexed by the generated snippet \hat c, although Q rows are codebase entities and the paper does not define a mapping from snippets to graph vertices or an extension of W to generated text; §3.2 only says W_ij 'measures the interaction strength' with no static-analysis procedure. These gaps make the dependency-consistency mechanism unexecutable as written, and the Table 4 ablations fail to control for the semantic-retrieval channel of Eq. (8), but they are correctness/validity threats, not cases where a claimed result equals its input by construction. The Jaccard metric (Eq. 10) is used both as an acceptance gate and in the Figure 2a diagnostic, yet it is not the dependent variable of the main claims. Hence no significant circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central method rests on several user-set hyperparameters and heuristic modeling assumptions; no new physical or mathematical entities are introduced.

free parameters (4)
  • dependency codebook levels q1..q8 = 1,2,3,4,5,6,7,8
    Hand-chosen integer levels for quantized strong dependencies; not learned from data.
  • low-rank dimension k = 8
    Set to default value; no sensitivity analysis reported for k.
  • regularization coefficients lambda_A, lambda_B = not reported
    Selected using validation performance but exact values absent; affects decomposition quality.
  • Jaccard consistency threshold tau = 0.8
    Tuned; sensitivity analysis shows peak near 0.8, but 0.3–0.9 all similar.
axioms (4)
  • domain assumption W adjacency matrix can be constructed from static code analysis and reflects true interaction strength.
    Introduced in §3.2; no details on how edges are weighted or how incomplete static analysis is handled.
  • ad hoc to paper Decomposition W ≈ Q + AB^T separates strong explicit from weak implicit dependencies.
    Core modeling assumption in Eq. 1; the separation is motivated qualitatively, not derived.
  • domain assumption Jaccard overlap between semantic and structural top-k retrievals indicates code correctness/consistency.
    Used in Eq. 10 as acceptance criterion; no theoretical or empirical justification beyond the sensitivity plot.
  • domain assumption Pre-trained code embeddings are suitable for semantic retrieval and align with the dependency matrix.
    Assumed in §3.4; no analysis of embedding quality or calibration.

pith-pipeline@v1.3.0-alltime-deepseek · 13272 in / 9733 out tokens · 81023 ms · 2026-08-01T20:10:57.847242+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Dependency-Guided Code Generation: Structured Matrix Decomposition and Consistency-Guided Refinement." pith.science (2026). https://pith.science/paper/2P6SSOOC

@misc{pith2026260716692,
  author       = {Pith},
  title        = {Pith review of: Dependency-Guided Code Generation: Structured Matrix Decomposition and Consistency-Guided Refinement},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2P6SSOOC}},
  note         = {Machine review of arXiv:2607.16692}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

The increasing complexity of modern software systems has made automated code generation a fundamental task in software engineering. However, existing approaches often fail to adequately capture the intricate, multi-level dependencies among code entities, leading to generated code that is logically incomplete or difficult to integrate into real-world systems. To address this limitation, we propose a dependency-aware code generation framework that explicitly models interactions among code entities through a graph-based representation. We decompose dependencies into two complementary components: a quantized matrix that captures strong, explicit relations, and a sparse low-rank factorization that models weaker, implicit interactions. The decomposition is efficiently learned via an alternating optimization procedure. During code generation, the learned dependency structure is incorporated as a constraint, ensuring both semantic coherence and structural consistency of the generated code. Furthermore, we introduce a sparse triplet representation for strong dependencies, significantly improving storage efficiency and computational scalability. Extensive experiments demonstrate that our approach consistently produces code with superior semantic alignment and structural fidelity compared to existing methods.

Figures

Figures reproduced from arXiv: 2607.16692 by Guangyuan Dong, Hao Zhang, Kai Shu, Mingqiao Mo, Wenhua Nie, Xiaodong Fan, Xin Xiao, Yangchen Zeng, Zhaolu Kang, Zikai Xiao.

Figure 1
Figure 1. Figure 1: Overview of the proposed dependency-aware code generation framework. Left: We first construct a semantic dependency graph from the input codebase, where nodes represent code entities (e.g., functions, classes, files) and edges encode their interactions, which are further summarized as an adjacency matrix capturing inter-entity relationships. Middle: The dependency matrix is decomposed into two complementar… view at source ↗
Figure 2
Figure 2. Figure 2: Comprehensive analysis of dependency-aware code generation, including dependency consistency across [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

126 extracted references · 1 canonical work pages

  1. [1]

    Abreu, Rui and Zoeteweij, Peter and van Gemund, Arjan J. C. , booktitle=. On the Accuracy of Spectrum-based Fault Localization , year=

  2. [2]

    2019 , eprint=

    Getafix: Learning to Fix Bugs Automatically , author=. 2019 , eprint=

    2019

  3. [3]

    Fuzzing: Challenges and Reflections , year=

    Boehme, Marcel and Cadar, Cristian and Roychoudhury, Abhik , journal=. Fuzzing: Challenges and Reflections , year=

  4. [4]

    Communications of the ACM , year=

    Symbolic execution for software testing: three decades later , author=. Communications of the ACM , year=

  5. [5]

    2021 , eprint=

    Evaluating Large Language Models Trained on Code , author=. 2021 , eprint=

    2021

  6. [6]

    Automated Repair of Programs from Large Language Models , year=

    Fan, Zhiyu and Gao, Xiang and Mirchev, Martin and Roychoudhury, Abhik and Tan, Shin Hwei , booktitle=. Automated Repair of Programs from Large Language Models , year=

  7. [7]

    Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis , year=

    Crash-avoiding program repair , author=. Proceedings of the 28th ACM SIGSOFT International Symposium on Software Testing and Analysis , year=

  8. [8]

    Automatic Program Repair , year=

    Le Goues, Claire and Pradel, Michael and Roychoudhury, Abhik and Chandra, Satish , journal=. Automatic Program Repair , year=

  9. [9]

    arXiv preprint arXiv:2203.05079 , year=

    Language Server Protocol and Implementation: Supporting Language-Smart Editing and Programming Tools , author=. arXiv preprint arXiv:2203.05079 , year=

    Pith/arXiv arXiv

  10. [10]

    Impact of Code Language Models on Automated Program Repair , year=

    Jiang, Nan and Liu, Kevin and Lutellier, Thibaud and Tan, Lin , booktitle=. Impact of Code Language Models on Automated Program Repair , year=

  11. [11]

    2023 , eprint=

    SWE-bench: Can Language Models Resolve Real-World GitHub Issues? , author=. 2023 , eprint=

    2023

  12. [12]

    and Yang, John and Wettig, Alexander and Yao, Shunyu and Pei, Kexin and Press, Ofir and Narasimhan, Karthik R

    Jimenez, Carlos E. and Yang, John and Wettig, Alexander and Yao, Shunyu and Pei, Kexin and Press, Ofir and Narasimhan, Karthik R. , booktitle=. 2024 , url=

    2024

  13. [13]

    PyAnalyzer: An Effective and Practical Approach for Dependency Extraction from Python Code , year=

    Jin, Wuxia and Xu, Shuo and Chen, Dawei and He, Jiajun and Zhong, Dinghong and Fan, Ming and Chen, Hongxu and Zhang, Huijia and Liu, Ting , booktitle=. PyAnalyzer: An Effective and Practical Approach for Dependency Extraction from Python Code , year=

  14. [14]

    Proceedings of the 24th International Conference on Software Engineering (ICSE) , year=

    Visualization of test information to assist fault localization , author=. Proceedings of the 24th International Conference on Software Engineering (ICSE) , year=

  15. [15]

    Software: Practice and Experience , year=

    An evaluation of pure spectrum‐based fault localization techniques for large‐scale software systems , author=. Software: Practice and Experience , year=

  16. [16]

    Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering , year=

    iFixR: bug report driven program repair , author=. Proceedings of the 2019 27th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering , year=

    2019

  17. [17]

    Science , year=

    Competition-level code generation with AlphaCode , author=. Science , year=

  18. [18]

    Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis , year=

    CoCoNuT: combining context-aware neural translation models using ensemble for program repair , author=. Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis , year=

  19. [19]

    SapFix: Automated End-to-End Repair at Scale , year=

    Marginean, Alexandru and Bader, Johannes and Chandra, Satish and Harman, Mark and Jia, Yue and Mao, Ke and Mols, Alexander and Scott, Andrew , booktitle=. SapFix: Automated End-to-End Repair at Scale , year=

  20. [20]

    Angelix: Scalable Multiline Program Patch Synthesis via Symbolic Analysis , year=

    Mechtaev, Sergey and Yi, Jooyong and Roychoudhury, Abhik , booktitle=. Angelix: Scalable Multiline Program Patch Synthesis via Symbolic Analysis , year=

  21. [21]

    2023 , eprint=

    Better Automatic Program Repair by Using Bug Reports and Tests Together , author=. 2023 , eprint=

    2023

  22. [22]

    SemFix: Program repair via semantic analysis , year=

    Nguyen, Hoang Duong Thien and Qi, Dawei and Roychoudhury, Abhik and Chandra, Satish , booktitle=. SemFix: Program repair via semantic analysis , year=

  23. [23]

    2022 , eprint=

    Trust Enhancement Issues in Program Repair , author=. 2022 , eprint=

    2022

  24. [24]

    2022 , eprint=

    Examining Zero-Shot Vulnerability Repair with Large Language Models , author=. 2022 , eprint=

    2022

  25. [25]

    , journal=

    Ryder, Barbara G. , journal=. Constructing the Call Graph of a Program , year=

  26. [26]

    2021 , eprint=

    PyCG: Practical Call Graph Generation in Python , author=. 2021 , eprint=

    2021

  27. [27]

    Journal of Software: Evolution and Process , year=

    CrossFix: Resolution of GitHub issues via similar bugs recommendation , author=. Journal of Software: Evolution and Process , year=

  28. [28]

    Automatically finding patches using genetic programming , year=

    Weimer, Westley and Nguyen, ThanhVu and Le Goues, Claire and Forrest, Stephanie , booktitle=. Automatically finding patches using genetic programming , year=

  29. [29]

    2023 , eprint=

    User-Centric Deployment of Automated Program Repair at Bloomberg , author=. 2023 , eprint=

    2023

  30. [30]

    Eric and Gao, Ruizhi and Li, Yihao and Abreu, Rui and Wotawa, Franz , journal=

    Wong, W. Eric and Gao, Ruizhi and Li, Yihao and Abreu, Rui and Wotawa, Franz , journal=. A Survey on Software Fault Localization , year=

  31. [31]

    2023 , eprint=

    Conversational Automated Program Repair , author=. 2023 , eprint=

    2023

  32. [32]

    2024 , eprint=

    SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering , author=. 2024 , eprint=

    2024

  33. [33]

    2022 , eprint=

    A Syntax-Guided Edit Decoder for Neural Program Repair , author=. 2022 , eprint=

    2022

  34. [34]

    IEEE Transactions on Software Engineering , year=

    GenProg: A Generic Method for Automatic Software Repair , author=. IEEE Transactions on Software Engineering , year=

  35. [35]

    Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages , year=

    Automatic patch generation by learning correct code , author=. Proceedings of the 43rd Annual ACM SIGPLAN-SIGACT Symposium on Principles of Programming Languages , year=

  36. [36]

    Proceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering (ASE) , year=

    An empirical investigation into learning bug-fixing patches in the wild via neural machine translation , author=. Proceedings of the 34th IEEE/ACM International Conference on Automated Software Engineering (ASE) , year=

  37. [37]

    Software Engineering Journal , year =

    John Smith , title =. Software Engineering Journal , year =

  38. [38]

    arXiv preprint arXiv:2009.03300 , year=

    Measuring Massive Multitask Language Understanding , author=. arXiv preprint arXiv:2009.03300 , year=

    Pith/arXiv arXiv 2009

  39. [39]

    Advances in Neural Information Processing Systems , volume=

    Program synthesis with large language models , author=. Advances in Neural Information Processing Systems , volume=

  40. [40]

    arXiv preprint arXiv:2302.07469 , year=

    Exploring large-scale pre-trained models in reasoning tasks: Methods and benchmarks , author=. arXiv preprint arXiv:2302.07469 , year=

    Pith/arXiv arXiv

  41. [41]

    arXiv preprint arXiv:2102.04664 , year=

    CodeXGLUE: A benchmark dataset and open challenge for code intelligence , author=. arXiv preprint arXiv:2102.04664 , year=

    Pith/arXiv arXiv

  42. [42]

    Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics , pages=

    BLEU: a method for automatic evaluation of machine translation , author=. Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics , pages=

  43. [43]

    Text Summarization Branches Out , pages=

    ROUGE: A package for automatic evaluation of summaries , author=. Text Summarization Branches Out , pages=

  44. [44]

    Transactions of the Association for Computational Linguistics , volume=

    Experts, errors, and context: A large-scale study of human evaluation for machine translation , author=. Transactions of the Association for Computational Linguistics , volume=

  45. [45]

    Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics , year=

    Task-Aware Evaluation of Machine Translation Output: A Fine-Grained Error Categorization , author=. Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics , year=

  46. [46]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics , year=

    FactScore: Measuring the factual consistency of abstractive text summarization outputs , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics , year=

  47. [47]

    arXiv preprint arXiv:2304.08491 , year=

    LongEval: Guidelines for Assessing Faithfulness in Long-Form Text Generation , author=. arXiv preprint arXiv:2304.08491 , year=

    Pith/arXiv arXiv

  48. [48]

    arXiv preprint arXiv:2304.10563 , year=

    Improving Evaluation of Long-Form Text Generation with Machine-Aided Analyses , author=. arXiv preprint arXiv:2304.10563 , year=

    Pith/arXiv arXiv

  49. [49]

    Proceedings of the 17th International Conference on Natural Language Generation , year=

    Evaluating GPT-4 for Text Generation: Coherence, Consistency, and Practical Applications , author=. Proceedings of the 17th International Conference on Natural Language Generation , year=

  50. [50]

    arXiv preprint arXiv:2303.09854 , year=

    On the Use of LLMs for Programming and Software Engineering Tasks: Benchmarking GPT-4 and Beyond , author=. arXiv preprint arXiv:2303.09854 , year=

    Pith/arXiv arXiv

  51. [51]

    Proceedings of the 35th International Conference on Software Engineering , year=

    Evaluating AI Systems in Real-World Software Development: A Comprehensive Analysis , author=. Proceedings of the 35th International Conference on Software Engineering , year=

  52. [52]

    Proceedings of the 42nd International Conference on Software Engineering , pages=

    Neural Program Repair in a Multi-Modal Embedding Space , author=. Proceedings of the 42nd International Conference on Software Engineering , pages=

  53. [53]

    Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing , pages=

    Efficient large-scale neural architecture search via parameter sharing , author=. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing , pages=

    2020

  54. [54]

    Proceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) , pages=

    Automated Program Repair Using Pre-Trained Transformers with Bug Understanding , author=. Proceedings of the 36th IEEE/ACM International Conference on Automated Software Engineering (ASE) , pages=

  55. [55]

    Proceedings of the 2018 International Conference on Software Engineering , pages=

    Program Repair via Data-Driven Code Transformation , author=. Proceedings of the 2018 International Conference on Software Engineering , pages=

    2018

  56. [56]

    Proceedings of the 44th IEEE/ACM International Conference on Software Engineering (ICSE) , year=

    Studying the Usage of Text-to-Text Transfer Transformer to Automate Software Repair , author=. Proceedings of the 44th IEEE/ACM International Conference on Software Engineering (ICSE) , year=

  57. [57]

    Proceedings of the 38th International Conference on Machine Learning (ICML) , pages=

    Graph-Based, Self-Supervised Program Repair from Diagnostic Feedback , author=. Proceedings of the 38th International Conference on Machine Learning (ICML) , pages=

  58. [58]

    IEEE Transactions on Software Engineering , volume=

    Automatic Software Repair: A Survey , author=. IEEE Transactions on Software Engineering , volume=

  59. [59]

    Proceedings of the 40th International Conference on Software Engineering (ICSE) , year=

    Context-Aware Patch Generation for Better Automated Program Repair , author=. Proceedings of the 40th International Conference on Software Engineering (ICSE) , year=

  60. [60]

    ACM Computing Surveys , volume=

    Automatic Software Repair: A Bibliography , author=. ACM Computing Surveys , volume=

  61. [61]

    Proceedings of the 37th International Conference on Software Engineering (ICSE) , pages=

    An Analysis of Patch Plausibility and Correctness for Generate-and-Validate Patch Generation Systems , author=. Proceedings of the 37th International Conference on Software Engineering (ICSE) , pages=

  62. [62]

    International Conference on Learning Representations (ICLR) , year =

    Xin Jin and John Doe and Jane Smith , title =. International Conference on Learning Representations (ICLR) , year =

  63. [63]

    Proceedings of the 30th International Conference on Machine Learning , series =

    Razvan Pascanu and Tomas Mikolov and Yoshua Bengio , title =. Proceedings of the 30th International Conference on Machine Learning , series =. 2013 , address =

    2013

  64. [64]

    Hinton and Simon Osindero and Yee-Whye Teh , title =

    Geoffrey E. Hinton and Simon Osindero and Yee-Whye Teh , title =. Neural Computation , volume =. 2006 , publisher =

    2006

  65. [65]

    Journal of Machine Learning Research , volume =

    Ang Li and Tze Meng Low and Minh Hoai Nguyen and Bryan Catanzaro , title =. Journal of Machine Learning Research , volume =. 2017 , publisher =

    2017

  66. [66]

    Dally , title =

    Jungwook Kim and Yong-Deok Kim and Song Han and Huizi Mao and William J. Dally , title =. International Conference on Learning Representations (ICLR) , year =

  67. [67]

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , year =

    Heechul Yang and Zhen Dong and Suyog Gupta and Carole-Jean Wu and Jungwook Choi , title =. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , year =

  68. [68]

    IEEE Transactions on Circuits and Systems I: Regular Papers , volume =

    Tianyun Zhang and Zhiyun Li and Di Wang and Ningyi Xu and Wenwen Feng and Jack Sampson and Vijaykrishnan Narayanan , title =. IEEE Transactions on Circuits and Systems I: Regular Papers , volume =. 2020 , publisher =

    2020

  69. [69]

    European Conference on Computer Vision (ECCV) , year =

    Mohammad Rastegari and Vicente Ordonez and Joseph Redmon and Ali Farhadi , title =. European Conference on Computer Vision (ECCV) , year =

  70. [70]

    IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , year =

    Yiwen Guo and Anbang Yao and Yurong Chen , title =. IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , year =

  71. [71]

    Golub and Charles F

    Gene H. Golub and Charles F. Van Loan , title =. 1996 , publisher =

    1996

  72. [72]

    IEEE Transactions on Software Engineering , volume =

    Feng Yu and Alvin Cheung and Armando Fox , title =. IEEE Transactions on Software Engineering , volume =. 2018 , publisher =

    2018

  73. [73]

    Biamonte and Qiang Qiu , title =

    Shangqian Shen and Ang Li and Yiyu Shi and Jacob D. Biamonte and Qiang Qiu , title =. NeurIPS , year =

  74. [74]

    Hagberg and Daniel A

    Aric A. Hagberg and Daniel A. Schult and Pieter J. Swart , title =. Proceedings of the 7th Python in Science Conference , year =

  75. [75]

    Ieee Access , volume=

    Code generation using machine learning: A systematic review , author=. Ieee Access , volume=. 2022 , publisher=

    2022

  76. [76]

    arXiv preprint arXiv:2207.10397 , year=

    Codet: Code generation with generated tests , author=. arXiv preprint arXiv:2207.10397 , year=

    Pith/arXiv arXiv

  77. [77]

    Proceedings of the 28th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering , pages=

    Intellicode compose: Code generation using transformer , author=. Proceedings of the 28th ACM joint meeting on European software engineering conference and symposium on the foundations of software engineering , pages=

  78. [78]

    2020 , url =

    NVIDIA TensorRT: Programmable Inference Accelerator , author =. 2020 , url =

    2020

  79. [79]

    Science , volume=

    Competition-level code generation with alphacode , author=. Science , volume=. 2022 , publisher=

    2022

  80. [80]

    Kwok , title =

    Jiechao Li and Zhaofeng Zhang and Yu Liu and James T. Kwok , title =. Journal of Machine Learning Research , volume =

Showing first 80 references.