Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

SemAgent: A Semantics Aware Program Repair Agent

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SemAgent, a deterministic workflow-based repair agent, argues that execution, issue, and code semantics—not more autonomy—produce complete patches, reporting 134 of 300 SWE-Bench Lite issues solved (44.66%), the highest among…

desk verdict Well-built pipeline with plausible internal ablations, but the leaderboard claim is confounded by preloaded reproduction tests and the 51.33% multi-agent number is an unrun union. read the letter →

arxiv 2506.16650 v1 pith:EFVJX332 submitted 2025-06-19 cs.SE cs.AIcs.MA

Anvith Pabba , Alex Mathai , Anindya Chakraborty , Baishakhi Ray This is my paper · ORCID
classification cs.SEcs.AIcs.MA
keywords automatedprogramrepairrepository-levelbugfixingLLMagentsworkflow-basedissuesemanticscodeexecutionSWE-benchLite
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Automated repair agents often stare at the most suspicious lines and patch only those, producing fixes that are overfitted to the issue's example or that miss consistency and edge cases. This paper argues the remedy is to make the pipeline semantics-aware: use execution behavior to localize the bug, generalize the issue into abstract directions, and trace the code's execution flows before generating fixes. The result is SemAgent, a deterministic workflow that solves 134 of 300 issues on the SWE-Bench Lite benchmark (44.66%), which the authors report as the highest among workflow-based methods and a 7.66-point gain over their SpecRover-style baseline. If the claim holds, it would mean that structured, interpretable workflows can outperform open-ended agentic systems on repository-level repair without fine-tuning.

What carries the argument

The load-bearing mechanism is a three-channel semantics model feeding a two-stage repair-and-review architecture. Execution semantics triangulate the buggy region through reproducer stack traces, execution-trace file lists, and SBFL suspiciousness rankings. Issue semantics is a generalized '<directions>' block extracted by prompting the LLM to abstract past the specific failing example. Code semantics is built from execution flows—high-level paths through the file, decomposed into steps and mapped to concrete code snippets through structured JSON. These feed a repair stage that proposes fine-grained fixes and a reviewer stage that filters to issue-relevant fixes; an aggregator then deduplicates, merges, and refines via an execution loop. In the paper's ablations, adding the repair stage raises the baseline from 37% to 42%, the reviewer adds another 2.66 points, and issue semantics alone accounts for 10 points on a 50-issue subset.

What would settle it

Re-run the published workflow baselines with the same preloaded reproduction tests and the same temperature-zero settings, and re-run SemAgent without those preloaded tests. If any baseline reaches 134 of 300 issues, or if SemAgent drops below 40.67% when the preloaded tests are removed, the paper's claim that it is the best workflow-based method on this benchmark would be contradicted.

Watch

Extended reading notes

Core claim

SemAgent is defined as a function $F(I, D, T) \to p$ that turns an issue, a codebase, and regression tests into a final patch. It first localizes with execution semantics—stack traces from an iteratively refined reproducer, files appearing in execution traces, and spectrum-based fault localization—then runs a standard patch generation, validation, and selection cycle to get an initial patch. The repair stage abstracts the issue into generalized directions (issue semantics), decomposes the buggy file's execution flows into natural-language steps mapped to code snippets (code semantics), and proposes fine-grained fixes. A reviewer agent filters fixes by relevance, and an aggregator merges them into a final patch. On 300 SWE-Bench Lite issues the paper reports 134 solved (44.66%), 7.66 points above its baseline, and attributes the gain to semantics plus the reviewer rather than to a stronger model.

Load-bearing premise

The headline comparison assumes it is fair to give SemAgent preloaded reproduction tests while taking baseline scores from earlier runs that did not have those tests.

Editorial extensions

If this is right

  • Deterministic workflow-based repair, using an off-the-shelf model, can exceed open-ended agentic systems on the benchmark if the semantic components are the source of the gain.
  • Adding issue-semantics abstraction to an existing Search-Edit-Test pipeline is a drop-in change that could transfer the method's benefits to other repair systems.
  • The reviewer stage's 2.66-point contribution suggests that filter-and-merge post-processing is an effective precision lever independent of generation quality.
  • The reported multi-agent overlap of 154 issues implies that different semantic views produce complementary fixes, so aggregating diverse semantic agents is a path toward higher solve rates without a stronger base model.
  • Execution semantics improve localization by 3.67 points overall and matter most for repositories with heavy setup, which guides where to invest in reproducer infrastructure.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: a clean test of the semantics thesis would run SemAgent with and without the preloaded reproduction tests; if the gains vanish, the contribution is test provisioning rather than semantic understanding.
  • Beyond the paper: because the method is deterministic and caches repair-stage results, its cost model (average $6.9 per issue) suggests a practical deployment could use cheaper models for localization and reserve the expensive model for the repair and review stages.
  • Beyond the paper: the single-file scope means multi-file consistency bugs are out of reach; extending the flow-to-snippet mapping to a repository-level call graph would be a direct testable extension of the code-semantics machinery.
  • Beyond the paper: the gap between the 44.66% single-run result and the 51.33% multi-agent overlap hints that a selection policy for choosing among candidate semantic fixes could recover most of that gap without new semantic signals.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces SemAgent, a workflow-based automated program repair agent that combines execution semantics (reproducer stack traces, execution traces, spectrum-based fault localization), issue semantics (an LLM-generated generalization of the issue statement), and code semantics (structured execution flows and step-level code mapping) to produce patches for SWE-bench Lite issues. The pipeline first localizes the bug using execution-semantics evidence, generates an initial patch via an APR workflow, then refines it through a repair stage that produces fine-grained fixes, a reviewer agent that filters relevant fixes, and a patch aggregator that merges and validates the final patch. The authors report a solve rate of 44.66% (134/300) on SWE-bench Lite, claim this is the highest among workflow-based methods, and present a multi-agent extension estimate of 51.33% (154/300) based on the union of ablated configurations. The paper includes ablations of the repair stage, issue semantics, reviewer agent, and execution-semantics-based localization.

Significance. If the 44.66% SWE-bench Lite result is established under a fair comparison, this is a meaningful advance for deterministic, workflow-based APR: it uses an out-of-the-box LLM with zero decoding temperature, requires no fine-tuning, and provides full prompts in the appendix, which supports reproducibility of the pipeline design. The paper explicitly targets the known failure mode of hyper-localized patches and provides a motivating example (astropy-14635) where SemAgent produces a more complete fix than typical overfitted patches. The ablation structure is sensible and the components are clearly described. However, the headline comparison is currently undermined by an evaluation-setup confound (preloaded reproduction tests provided only to SemAgent and not to baselines), and the multi-agent 51.33% figure is presented as a result despite being an untested union of separate runs. The issue-semantics ablation also includes the motivating example used to design the prompt, introducing a circularity that weakens the causal claim about issue semantics.

major comments (4)
  1. [§5.2, §5.3, Table 1] The headline comparison against Agentless (40.67%), SpecRover (37%), and OpenHands (41.67%) is confounded by the preloaded Amazon Q Developer reproduction tests. Section 5.2 states that SemAgent takes as input the issue description, codebase, and preloaded Amazon Q Developer reproduction tests, while Section 5.3 reports that the baseline numbers are taken from prior papers and the baselines were not re-run with these preloaded tests. Since Section 4.1 uses reproducer crash traces and spectrum-based fault localization, these tests can directly influence localization and patch validation, providing SemAgent with information about expected behavior that the cited baselines did not receive. The abstract's claim of beating all workflow-based approaches therefore is not supported by the current experimental design. The authors should either re-run the baseline methods with the same preloaded tests, or ablate SemAgent without the preloaded tests, and report both comparisons.
  2. [§6.2, Table 3 (RQ2)] The issue-semantics ablation is circular because it includes the very example used to design the issue-semantics prompt. Section 3.1 introduces astropy-14635 as the motivating example underlying the design, and Section 6.2 states that Table 3 evaluates issue semantics on '49 randomly selected low-cost issues along with the motivating example.' Including this example in the 50-issue ablation means the prompt was effectively tuned on a test point, which can inflate the observed 60% vs. 50% difference and undermines the claim that issue semantics provides a causal improvement. The ablation should be re-run on the 49 randomly selected issues alone, excluding astropy-14635, to establish that the benefit generalizes.
  3. [§6.1, Table 1, Figure 3] The reported 51.33% for SemAgent-Multi is not an evaluated system; it is the size of the union of issues solved by different ablation configurations, as shown in Figure 3. Table 1 lists SemAgent-Multi as a workflow-based approach with a 51.33% solve rate, but Section 6.1 only says that 'extending our aggregator to a multi-agent setup, we could potentially solve up to 154 issues.' Presenting this untested upper bound in the main leaderboard table as if it were a measured result is misleading. The multi-agent configuration should be actually run and evaluated, or the table should clearly label the 154-issue figure as an upper-bound estimate rather than a solved rate.
  4. [§6.1, §8] The evaluation is a single run with no released patches (Section 6.1: 'All results are from a single run'), and the full system relies on a proprietary model (Claude Sonnet 3.7). While temperature zero and a deterministic workflow reduce variance, LLM-based steps can still vary across runs, and the absence of any variance estimate or multiple-run data makes the 44.66% point estimate hard to assess for statistical significance. The authors should provide at least a small multiple-run study on a random subset (e.g., 50 issues) with confidence intervals, and release the generated patches for the 300 issues to enable independent verification.
minor comments (6)
  1. [§5.3] The text refers to 'SpecRover (Zhang et al., 2024)' but the reference list entry is Ruan et al. (2024); the citation should be corrected.
  2. [§6.2, Table 3] Table 3 is titled 'Random Set of 50 SWEBench Issues', but the text says the set is '49 randomly selected low-cost issues along with the motivating example'; the title and text should be made consistent, and the inclusion of a non-random example should be disclosed in the table itself.
  3. [Figure 3 caption] The caption 'Overlap of Ablations - 154 issues resolved in total' should clarify that this is the union of issues resolved by different ablation variants, not by a single system, to avoid confusion with the table entry.
  4. [§2] There are typos such as 'acctute' and 'repostiory-level' in the first paragraph of Related Work; the text should be proofread.
  5. [§5.2] The statement that SemAgent 'uses no test-specific metadata or hints' is in tension with the preloaded reproduction tests being an input; the authors should clarify the intended distinction between 'test-specific metadata' and 'reproduction tests' to avoid an apparent contradiction.
  6. [§8] The limitation that the current implementation is restricted to single-file fixes is acknowledged, but it would be helpful to report how many of the 300 SWE-bench Lite issues involve multi-file gold patches, since the headline result may partly reflect the benchmark composition.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor circularity in the issue-semantics ablation: the motivating example used to design the prompt is also included in the validation set; the headline 44.66% benchmark result is not circular.

  1. other [Section 3.1 (Motivation) and Section 6.2 / Table 3 (RQ2 ablation)]
    "Table 3 evaluates the impact of incorporating issue semantics in the Repair Stage of SemAgent , using 49 randomly selected low-cost issues along with the motivating example. The results reveal that 5 issues remained unresolved when issue semantics were excluded. ... Notably, in the motivating example, the absence of semantic context leads to inadequate handling of consistency and edge cases showing the critical role issue semantics play in SemAgent ’s pipeline."

    The astropy-14635 example is used in Section 3.1 to motivate and design SemAgent's issue-semantics abstraction ('we introduce a prompting step that guides the LLM to abstract beyond surface artifacts'), and the same example is then explicitly added to the 50-issue ablation in Table 3 that is used to validate issue semantics. The paper even highlights that this particular example is solved only with issue semantics. Thus the component-level claim is not tested out-of-sample: one of the strongest pieces of evidence is drawn from the design input itself. The full 300-issue SWE-Bench Lite result in RQ1 does not depend on this 50-issue ablation, so the circularity is local and low severity.

full rationale

SemAgent's central claim (44.66% on SWE-Bench Lite) is a measured result on an external benchmark; it is not derived by construction from the issue-semantics prompt, nor is any fitted parameter renamed as a prediction. The 7.66% improvement is arithmetic against the published SpecRover baseline, and the RQ2/RQ3 ablations are empirical comparisons rather than deductions from their own inputs. The only step that approaches circularity is local: the astropy-14635 motivating example in Section 3.1 is used to motivate and design the issue-semantics prompting, and the same example is then deliberately included in the 50-issue Table 3 ablation used to validate issue semantics; the caption's 'Random Set' wording is also inconsistent with the text's admission that the motivating example was added. This weakens the component-level claim but does not affect the full 300-issue result. The preloaded Amazon Q reproduction tests described in Section 5.2 are an evaluation-fairness concern (the cited baselines were not run with those tests), but they are an input to the system rather than a circular derivation of the output score. Accordingly, the circularity score is low.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The paper's claims rest on pipeline hyperparameters chosen by hand, on the benchmark oracle, and on assumed fairness of giving the agent preloaded reproduction tests. The free parameters are not fitted to SWE-bench test labels; they are engineering choices that nevertheless affect the reported performance and should be audited in replication.

free parameters (6)
  • top_suspicious_files_from_trace = 7
    Appendix A.2: the ordered suspicious-file list from execution traces is truncated to the top 7 files for the localization agent; this cutoff is chosen by hand and affects localization input.
  • top_suspicious_functions_from_sbfl = 5
    Appendix A.2: spectrum-based fault localization yields the top 5 suspicious functions, which are sent to the localization agent; a hand-chosen cutoff.
  • pipeline_retries = 3
    Section 5.4: up to 3 pipeline retries per issue; an engineering choice that affects success rate and cost.
  • localization_rounds = 15
    Section 5.4: up to 15 localization rounds per issue; a hand-chosen budget for the localization agent.
  • reproducer_attempts = 10
    Section 5.4: up to 10 reproducer attempts per issue; a hand-chosen budget for generating a failing test script.
  • decoding_temperature = 0
    Section 5.4: temperature fixed to zero to reduce variance and improve caching; LLM APIs do not guarantee bitwise determinism, so this choice affects reproducibility claims.
assumptions (4)
  • domain assumption SWE-bench Lite automated test suite is a valid oracle for issue resolution.
    The paper equates solved with passing the benchmark's hidden tests; this is standard but assumes the tests capture developer intent.
  • ad hoc to paper Amazon Q Developer reproduction tests do not encode the gold patch or expected behavior beyond what a developer would know from the issue.
    Section 5.2 says the agent receives preloaded reproduction tests as input; the paper does not justify that this does not leak test information, which would differentially favor SemAgent over baselines that must generate their own reproducers.
  • domain assumption At temperature 0, Claude Sonnet 3.7 output is deterministic across runs.
    Section 5.4 claims zero temperature ensures stability and reproducibility; LLM APIs do not guarantee bitwise determinism, and the paper reports all results from a single run.
  • ad hoc to paper The motivating astropy-14635 example is representative of SWE-bench Lite issues, and its inclusion in the 50-issue ablation does not bias the issue-semantics result.
    Section 6.2 and Table 3 use 49 randomly selected low-cost issues along with the motivating example; the motivating example was used to design the issue-semantics prompt, so validating on it is partially circular.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SemAgent: A Semantics Aware Program Repair Agent." pith.science (2026). https://pith.science/paper/EFVJX332

@misc{pith2026250616650,
  author       = {Pith},
  title        = {Pith review of: SemAgent: A Semantics Aware Program Repair Agent},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EFVJX332}},
  note         = {Machine review of arXiv:2506.16650}
}
read the original abstract

Large Language Models (LLMs) have shown impressive capabilities in downstream software engineering tasks such as Automated Program Repair (APR). In particular, there has been a lot of research on repository-level issue-resolution benchmarks such as SWE-Bench. Although there has been significant progress on this topic, we notice that in the process of solving such issues, existing agentic systems tend to hyper-localize on immediately suspicious lines of code and fix them in isolation, without a deeper understanding of the issue semantics, code semantics, or execution semantics. Consequently, many existing systems generate patches that overfit to the user issue, even when a more general fix is preferable. To address this limitation, we introduce SemAgent, a novel workflow-based procedure that leverages issue, code, and execution semantics to generate patches that are complete - identifying and fixing all lines relevant to the issue. We achieve this through a novel pipeline that (a) leverages execution semantics to retrieve relevant context, (b) comprehends issue-semantics via generalized abstraction, (c) isolates code-semantics within the context of this abstraction, and (d) leverages this understanding in a two-stage architecture: a repair stage that proposes fine-grained fixes, followed by a reviewer stage that filters relevant fixes based on the inferred issue-semantics. Our evaluations show that our methodology achieves a solve rate of 44.66% on the SWEBench-Lite benchmark beating all other workflow-based approaches, and an absolute improvement of 7.66% compared to our baseline, which lacks such deep semantic understanding. We note that our approach performs particularly well on issues requiring multi-line reasoning (and editing) and edge-case handling, suggesting that incorporating issue and code semantics into APR pipelines can lead to robust and semantically consistent repairs.

Figures

Figures reproduced from arXiv: 2506.16650 by the authors.

Figure 1
Figure 1. Motivating Example - 1 is the issue description, 2 is the developer patch & SemAgent patch, 3 and 4 are incomplete/overfitted patches, 5 is the SemAgent generated issue semantics, 6 is the SemAgent generated code semantics. 2025; Mathai et al., 2024) have made it possible to conduct agent-related research by leveraging exe￾cution feedback through the automatic evaluation of test-cases. These benchmarks have conseque… view at source ↗
Figure 2
Figure 2. Pipeline of SemAgent . It takes as input a codebase, GitHub issue, and regression tests. First, it uses execution semantics 1 to localize the buggy code 2 and apply a standard APR workflow—patch generation 3 , validation 4 , and selection 5 —producing a candidate patch, which is often incomplete or inconsistent. Next, SemAgent refines this patch using issue 6 and code semantics 8 , extracted from the buggy file 7 an… view at source ↗
Figure 3
Figure 3. Overlap of Ablations - 154 issues resolved in [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MultiFixer: A Coordinator-Proposer Based Multi-Agent Framework For Fixing Multi-Hunk Bugs

    cs.SE 2026-07 conditional novelty 6.0 of 10

    Coordinator-proposer multi-agent repair schedules hunks, proposes candidate patches in parallel, and selects/refines them, fixing 326/835 Defects4J bugs with GPT-3.5 and 420 with Claude-3.5-Sonnet.

Reference graph

Works this paper leans on

29 extracted references · 7 canonical work pages · cited by 1 Pith paper

  1. [1]

    https://aws.amazon.com/about-aws/whats-new/2025/04/amazon-q-developer-releases-state-art-agent-feature-development/ Amazonqdeveloper

    2025. https://aws.amazon.com/about-aws/whats-new/2025/04/amazon-q-developer-releases-state-art-agent-feature-development/ Amazonqdeveloper

    work page 2025

  2. [2]

    https://github.com/astropy/astropy Astropy

    2025. https://github.com/astropy/astropy Astropy

    work page 2025

  3. [3]

    https://www.cognition.ai/blog/introducing-devin Devin

    2025. https://www.cognition.ai/blog/introducing-devin Devin

    work page 2025

  4. [4]

    https://www.anthropic.com/engineering/building-effective-agents Types of agents

    2025. https://www.anthropic.com/engineering/building-effective-agents Types of agents

    work page 2025

  5. [5]

    Rui Abreu, Peter Zoeteweij, and Arjan JC Van Gemund. 2007. On the accuracy of spectrum-based fault localization. In Testing: Academic and industrial conference practice and research techniques-MUTATION (TAICPART-MUTATION 2007), pages 89--98. IEEE

    work page 2007

  6. [6]

    Vaibhav Aggarwal, Ojasv Kamal, Abhinav Japesh, Zhijing Jin, and Bernhard Sch \"o lkopf. 2025. Dars: Dynamic action re-sampling to enhance coding agent performance by adaptive tree traversal. arXiv preprint arXiv:2503.14269

    arXiv 2025

  7. [7]

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 2021. https://arxiv.org/abs/2107.03374 Evaluating large lang...

    arXiv 2021

  8. [8]

    Xiang Gao, Yannic Noller, and Abhik Roychoudhury. 2022. https://arxiv.org/abs/2211.12787 Program repair . Preprint, arXiv:2211.12787

    arXiv 2022

Show all 29 references
  1. [9]

    Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan

    Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. 2024. https://arxiv.org/abs/2310.06770 Swe-bench: Can language models resolve real-world github issues? Preprint, arXiv:2310.06770

    2024 arXiv

  2. [10]

    Matthew Jin, Syed Shahriar, Michele Tufano, Xin Shi, Shuai Lu, Neel Sundaresan, and Alexey Svyatkovskiy. 2023. https://doi.org/10.1145/3611643.3613892 Inferfix: End-to-end program repair with llms . In Proceedings of the 31st ACM Joint European Software Engineering Conference ...

    2023

  3. [11]

    Sungmin Kang, Juyeon Yoon, and Shin Yoo. 2023. https://arxiv.org/abs/2209.11515 Large language models are few-shot testers: Exploring llm-based general bug reproduction . Preprint, arXiv:2209.11515

    2023 arXiv

  4. [12]

    Hongwei Li, Yuheng Tang, Shiqi Wang, and Wenbo Guo. 2025. Patchpilot: A stable and cost-efficient agentic patching framework. arXiv preprint arXiv:2502.02747

    2025 arXiv

  5. [13]

    Zhiyu Li, Shuai Lu, Daya Guo, Nan Duan, Shailesh Jannu, Grant Jenks, Deep Majumder, Jared Green, Alexey Svyatkovskiy, Shengyu Fu, and Neel Sundaresan. 2022. https://arxiv.org/abs/2203.09095 Automating code review activities by large-scale pre-training . Preprint, arXiv:2203.09095

    2022 arXiv

  6. [14]

    Alex Mathai, Chenxi Huang, Petros Maniatis, Aleksandr Nogikh, Franjo Ivan c i \'c , Junfeng Yang, and Baishakhi Ray. 2024. Kgym: A platform and dataset to benchmark large language models on linux kernel crash resolution. Advances in Neural Information Processing Systems, 37:78...

    2024

  7. [15]

    Muhammad Shihab Rashid, Christian Bock, Yuan Zhuang, Alexander Buchholz, Tim Esler, Simon Valentin, Luca Franceschi, Martin Wistuba, Prabhu Teja Sivaprasad, Woo Jung Kim, Anoop Deoras, Giovanni Zappella, and Laurent Callot. 2025. https://arxiv.org/abs/2504.08703 Swe-polybench:...

    2025 arXiv

  8. [16]

    Haifeng Ruan, Yuntong Zhang, and Abhik Roychoudhury. 2024. https://arxiv.org/abs/2408.02232 Specrover: Code intent extraction via llms . Preprint, arXiv:2408.02232

    2024 arXiv

  9. [17]

    Gabriel Ryan, Siddhartha Jain, Mingyue Shang, Shiqi Wang, Xiaofei Ma, Murali Krishna Ramanathan, and Baishakhi Ray. 2024. https://doi.org/10.1145/3643769 Code-aware prompting: A study of coverage-guided test generation in regression setting using llm . Proc. ACM Softw. Eng., 1(FSE)

    2024 doi

  10. [18]

    Weisong Sun, Yun Miao, Yuekang Li, Hongyu Zhang, Chunrong Fang, Yi Liu, Gelei Deng, Yang Liu, and Zhenyu Chen. 2024. https://arxiv.org/abs/2407.07959 Source code summarization in the era of large language models . Preprint, arXiv:2407.07959

    2024 arXiv

  11. [19]

    Michele Tufano, Dawn Drain, Alexey Svyatkovskiy, Shao Kun Deng, and Neel Sundaresan. 2021. https://arxiv.org/abs/2009.05617 Unit test case generation with transformers and focal context . Preprint, arXiv:2009.05617

    2021 arXiv

  12. [20]

    Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. 2024 a . https://arxiv.org/abs/2402.01030 Executable code actions elicit better llm agents . Preprint, arXiv:2402.01030

    2024 arXiv

  13. [21]

    Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H

    Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, and 5 others. 2025. https:...

    2025 arXiv

  14. [22]

    Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, and 1 others. 2024 b . Openhands: An open platform for ai software developers as generalist agents. In The Thirteenth International Conference on Lea...

    2024

  15. [23]

    Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. 2024. https://arxiv.org/abs/2407.01489 Agentless: Demystifying llm-based software engineering agents . Preprint, arXiv:2407.01489

    2024 arXiv

  16. [24]

    Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press

    John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. 2024. https://arxiv.org/abs/2405.15793 Swe-agent: Agent-computer interfaces enable automated software engineering . Preprint, arXiv:2405.15793

    2024 arXiv

  17. [25]

    Daoguang Zan, Zhirong Huang, Wei Liu, Hanwu Chen, Linhao Zhang, Shulin Xin, Lu Chen, Qi Liu, Xiaojian Zhong, Aoyan Li, Siyao Liu, Yongsheng Xiao, Liangqiang Chen, Yuyu Zhang, Jing Su, Tianyu Liu, Rui Long, Kai Shen, and Liang Xiang. 2025. https://arxiv.org/abs/2504.02605 Multi...

    2025 arXiv

  18. [26]

    Yuntong Zhang, Haifeng Ruan, Zhiyu Fan, and Abhik Roychoudhury. 2024. https://arxiv.org/abs/2404.05427 Autocoderover: Autonomous program improvement . Preprint, arXiv:2404.05427

    2024 arXiv

  19. [27]

    Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. 2024. https://arxiv.org/abs/2310.04406 Language agent tree search unifies reasoning acting and planning in language models . Preprint, arXiv:2310.04406

    2024 arXiv

  20. [28]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  21. [29]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.