REVIEW 30 cited by
AutoCodeRover: Autonomous Program Improvement
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
abstract
Researchers have made significant progress in automating the software development process in the past decades. Recent progress in Large Language Models (LLMs) has significantly impacted the development process, where developers can use LLM-based programming assistants to achieve automated coding. Nevertheless, software engineering involves the process of program improvement apart from coding, specifically to enable software maintenance (e.g. bug fixing) and software evolution (e.g. feature additions). In this paper, we propose an automated approach for solving GitHub issues to autonomously achieve program improvement. In our approach called AutoCodeRover, LLMs are combined with sophisticated code search capabilities, ultimately leading to a program modification or patch. In contrast to recent LLM agent approaches from AI researchers and practitioners, our outlook is more software engineering oriented. We work on a program representation (abstract syntax tree) as opposed to viewing a software project as a mere collection of files. Our code search exploits the program structure in the form of classes/methods to enhance LLM's understanding of the issue's root cause, and effectively retrieve a context via iterative search. The use of spectrum-based fault localization using tests, further sharpens the context, as long as a test-suite is available. Experiments on SWE-bench-lite (300 real-life GitHub issues) show increased efficacy in solving GitHub issues (19% on SWE-bench-lite), which is higher than the efficacy of the recently reported SWE-agent. In addition, AutoCodeRover achieved this efficacy with significantly lower cost (on average, $0.43 USD), compared to other baselines. We posit that our workflow enables autonomous software engineering, where, in future, auto-generated code from LLMs can be autonomously improved.
Forward citations
Cited by 30 Pith papers
-
Don't Blame the Large Language Model: How Agent Harness Evolution Shapes Coding Agent Quality
Across 35 Qwen Code releases with a fixed LLM, resolve rates do not significantly improve while token use and tool calls roughly double, driven by feature-heavy releases and high-risk components.
-
LoopsBench: From Harness Engineering to Loop Engineering in Coding Agent Evaluation
LoopsBench exposes coding agents to dependency-DAG tasks with regression obligations; the best configuration, Opus-4.7 with Claude Code and outer continuation, resolves 25.00% of tasks.
-
Preventing Premature Commitment in Coding Agents with an Evidence-Conditioned Execution Layer
ECLoop gates coding agents' commit actions on task-specific evidence conditions, raising SWE-bench Verified Pass@1 by 4.8-11.8 points without retraining or scaffold changes.
-
Semantic Drift in Bug Resolution: How Behavioral Signals Propagate from Reports to Tests and Patches
Behavioral signals in bug reports propagate only partially into tests and fixes; alignment is measurable but representation-dependent, and LLM judges are systematically optimistic versus human ratings.
-
LLM Agents Can See Code Repositories
Visual graphs of repository structure added to text inputs for multimodal LLM agents reduce token consumption by up to 26% while maintaining or improving issue-resolution accuracy.
-
Adversarial Reasoning for Repair Based on Inferred Program Intent
AdverIntent-Agent infers multiple adversarial program intents, generates tests for each, and produces patches, correctly repairing 77 Defects4J and 105 HumanEval-Java bugs.
-
COFFE: A Code Efficiency Benchmark for Code Generation
A new benchmark, COFFE, uses stressful test cases and CPU instruction counts to show LLM-generated code is often correct but time-inefficient.
-
Evaluating LLM-Based Regression Test Generation
A feedback-driven LLM tool generates regression tests for commits in minutes and, when its outputs seed a fuzzer, finds more bugs than a 24-hour state-of-the-art fuzzer on the same benchmark.
-
When Does Restricting a Coding Agent to execute_code Help? A Regime $\times$ Agent-Design Ablation
Code-only tool surfaces beat or match the cheapest tool-rich rival on cost in three of four (regime, agent) cells at matched pass rates; Claude on SWE-bench is the directional exception via failure cost.
-
Independent Patch Verification for Coding Agents with a Bidirectional Reconstruct-and-Verify Framework
RETRACE improves coding-agent patch correctness by having the model guess the problem a patch solves without seeing the original report, then reconciling that guess with the report to decide whether to submit or revise.
-
Coupling Planning with Episodic Memory in LLM Agents for Software Issue Resolution
Coupling a hierarchical phase planner with episodic memory lets an LLM repair agent hold onto its diagnosis and recover from failed edits, resolving 5.0pp more SWE-bench Verified issues.
-
Shared Organizational Memory for Enterprise Coding Agents: System Design and Deployment Snapshot
A production deployment snapshot shows hook-based automatic capture and LLM curation turning 900 agent learnings into 1,144 shared question-answer memories, with no evidence yet of retrieval or coding-task benefit.
-
SCOPE: Synthetic Conditional Objectives for Policy Evolution in Black-Box Combinatorial Optimization
SCOPE evolves LLM-generated auxiliary objective functions and selects a validated portfolio of them to guide fixed combinatorial search engines under strict black-box query budgets.
-
Agentic Program Repair from Test Failures at Scale: A Neuro-symbolic approach with static analysis and test execution feedback
Meta's Engineering Agent solved 42.3% of curated internal test-failure tasks offline and landed 25.5% of generated fixes in production over three months.
-
PAGENT: Learning to Patch Software Engineering Agents
LLM repair agents fail mostly on type and code-context problems, and a hybrid static-analysis/LLM tool fixes 29 of 127 previously failed type-related patches.
-
kAgent: An execution-guided crash resolution agent for the Linux kernel
An LLM agent with execution-trace-guided hypothesis generation resolves about 65% of fuzzed Linux kernel crashes when told which file to fix, but most of its 'fixed' crashes are resolved by deleting or bypassing code.
-
CSR-Bench: Benchmarking LLM Agents in Deployment of Computer Science Research Repositories
CSR-Bench and CSR-Agents show that LLM agents can complete under half of setup and data-download steps, and between 15 and 29 percent of training, inference, and evaluation steps, on 100 research repositories.
-
Large Language Model Critics for Execution-Free Evaluation of Code Changes
Reference-aware, test-centric LLM critics predict per-test outcomes of code patches and aggregate them into execution-free build status predictions on SWE-bench-Lite.
-
A Real-World Benchmark for Evaluating Fine-Grained Issue Solving Capabilities of Large Language Models
FAUN-Eval is a curated benchmark of 300 real GitHub issue-PR pairs that scores LLMs separately on question answering, fault localization, and code editing.
-
AEGIS: An Agent-based Framework for General Bug Reproduction from Issue Descriptions
AEGIS combines concise context extraction with finite-state-machine feedback control to make LLM agents reproduce more software bugs from issue descriptions than existing baselines.
-
Is Inter-Seed Cross-Play Enough? Evaluating the Robustness of Zero-Shot Coordination Algorithms to Implementation Details
For Other-Play in Yokai, agents trained with different implementation details coordinate across implementations about as well as across seeds, supporting inter-seed cross-play as a proxy for cross-implementation evaluation.
-
Agent Team Work Zone: An Automated, Persistent Workspace for Long-Lived Claude Code Agent Teams
A design and open-source implementation of a filesystem layer that persists Claude Code agent-team state to disk, enabling checkpointing, recovery, and cross-session communication.
-
ContextSniper: AntTrail's Token-Efficient Code Memory for Repository-Level Program Repair
ContextSniper reduces token use by 38.9-51.5% in repository-level program repair agents on SWE-bench Lite with 2 percentage point drops in resolution rate.
-
Detection Time Distribution Predicted Using Absorbing Boundary Conditions and Imaginary Potentials
Three inequivalent quantum detection-time proposals yield waveguide arrival distributions showing partial reflection, spin-orientation independence (for tested parameters), and width dependence when spin couples to th...
-
SemAgent: A Semantics Aware Program Repair Agent
SemAgent, a workflow-based repair agent using execution, issue, and code semantics, reports solving 134 of 300 SWE-bench Lite issues (44.66%).
-
CoRet: Improved Retriever for Code Editing
CoRet fine-tunes a code encoder with a repository-level likelihood loss and call-graph context, improving retrieval recall on SWE-bench and Long Code Arena.
-
Self-Evolving Coding Agents
A systematic survey that defines self-evolving coding agents, organizes systems into a five-category object-centered taxonomy, and analyzes when and on what evidence these agents evolve.
-
Repeton: Structured Bug Repair with ReAct-Guided Patch-and-Test Cycles
Repeton, a retrieval-free, fully open-source LLM agent, fixes 11.67% of SWE-bench Lite bugs through iterative structured patch-and-test cycles.
-
Agentic AI Systems Applied to tasks in Financial Services: Modeling and model risk management crews
A CrewAI-based multi-agent system with human oversight built financial models and carried out model risk management checks on three public credit datasets, with results comparable to AutoML and Kaggle baselines.
-
How is Google using AI for internal code migrations?
An experience report shows Google using fine-tuned LLMs plus AST-based tooling and human review to accelerate large internal code migrations, with estimated time savings of roughly 50% to 89%.
Discussion (0). Continue with ORCID to comment.