AIDev is a new open dataset of 456k AI-agent pull requests showing agents submit code faster than humans but with lower acceptance rates and simpler changes.
hub Canonical reference
Devanbu, Christoph Treude, and Michael Pradel
Canonical reference. 77% of citing Pith papers cite this work as background.
26
Pith papers citing it
Background
77% of classified citations
hub tools
citation-role summary
background 11
dataset 2
citation-polarity summary
representative citing papers
Defines documentation-to-code equivalence and introduces Documentary to generate matching docs for 53.4% of function snippets, raising LLM output prediction accuracy by 12.8-24.5% over human-written docs.
DateSAT introduces the first solver for satisfiability constraints over dates and calendar periods via five reduction strategies to integer SMT.
ReproBreak provides 449 verified locator breaks from real web test commits along with scripts to reproduce them automatically.
CodeQL detected 171 CVEs total, with 83 caught by a prior version before the fix; detections were often actionable within the vulnerable file but not stable across tool versions.
SmellBench is the first benchmark showing LLM agents resolve 47.7% of architectural code smells while accurately spotting false positives, but aggressive repairs often introduce new smells and degrade overall quality.
RESTestBench shows that LLM-generated REST API test effectiveness drops when interacting with faulty or mutated code, especially for vague requirements, indicating that high-detail requirements make direct SUT interaction unnecessary.
Software engineering scope expands beyond executable code to semi-executable artifacts best diagnosed by the new six-ring Semi-Executable Stack model.
AI agents modify logging less often than humans in 58.4% of repositories but produce higher log density when they change it; explicit logging instructions are rare (4.7%) and ignored 67% of the time, with humans performing 72.5% of post-generation log repairs.
The paper delivers a taxonomy of seven LLM study types in software engineering along with eight guidelines that separate mandatory requirements from recommended practices to address reproducibility challenges.
Re-evaluating four LLM code-efficiency benchmarks with 30-run statistical testing shows 93.89% of 'performant' implementations are indistinguishable from baselines; a multi-agent test-generation framework reveals hidden significant improvements in ~24% of previously non-significant tasks.
Babbling Suppression stops LLM code generation upon test passage to reduce token output and energy consumption by up to 65% across Python and Java benchmarks.
Case study of 18,020 Kubernetes PRs shows label-diff congruence is prevalent and stable, with higher congruence linked to fewer review participants among core developers and more among one-time contributors.
Classport embeds Maven dependency coordinates into Java class files as runtime annotations and uses a Java agent to report the dependencies actually executed during a run.
HiLSVA shows that a human-in-the-loop LLM agent system can help novices and experts complete scientific visualization tasks, while human oversight adds measurable execution time.
Developers most frequently reference the full Log4j migration guide in pull request descriptions (82.81% of cases) and continue consulting it during post-update maintenance tasks.
Documentation on testing in 160 OSS repositories shows a weak positive correlation (ρ=0.36) with higher test engagement ratios in pull requests, strengthening to moderate in high-activity repos.
A feasibility study showing SBOM graphs can support component-vulnerability classification and CVE-pair co-exploitation ranking, with acknowledged limits on leakage and generalization.
XARP provides a WebSocket-based remote-procedure system that lets Python code and AI agents control Unity XR clients, with benchmarks and user studies showing faster iteration than conventional XR workflows.
Specificity and Context predict actionable code generation while Verification predicts adoption and Context predicts integration depth in LLM-assisted PR workflows.
Presents a framework for generating cryptographically signed CSAF VEX advisories for agentic AI by binding SBOM/AIBOM artifacts to deterministic environment capture and runtime telemetry.
The paper outlines a controlled study protocol using staged programming tasks to measure the effects of configuration mechanisms on build-versus-buy decisions in Claude Code and OpenAI Codex.
LLMs reach moderate macro-F1 scores of 0.36-0.37 when classifying code review comments into six smells and three useful intents, with one-shot examples helping some models on intent labels.
The paper reviews energy-aware computing literature and constructs a taxonomy organized by hardware/software aspects, measurement, optimizations, scheduling, scaling, consolidation, federated learning, and cooling.
citing papers explorer
-
The Rise of AI Teammates in Software Engineering (SE) 3.0: How Autonomous Coding Agents Are Reshaping Software Engineering
AIDev is a new open dataset of 456k AI-agent pull requests showing agents submit code faster than humans but with lower acceptance rates and simpler changes.
-
Natural Language-Focused Software Engineering via Code-Documentation Equivalence
Defines documentation-to-code equivalence and introduces Documentary to generate matching docs for 53.4% of function snippets, raising LLM output prediction accuracy by 12.8-24.5% over human-written docs.
-
DateSAT: A Framework for Solving Date and Period Constraints
DateSAT introduces the first solver for satisfiability constraints over dates and calendar periods via five reduction strategies to integer SMT.
-
ReproBreak: A Dataset of Reproducible Web Locator Breaks
ReproBreak provides 449 verified locator breaks from real web test commits along with scripts to reproduce them automatically.
-
Longitudinal Analyses of SAST Tools: A CodeQL Case Study
CodeQL detected 171 CVEs total, with 83 caught by a prior version before the fix; detections were often actionable within the vulnerable file but not stable across tool versions.
-
SmellBench: Evaluating LLM Agents on Architectural Code Smell Repair
SmellBench is the first benchmark showing LLM agents resolve 47.7% of architectural code smells while accurately spotting false positives, but aggressive repairs often introduce new smells and degrade overall quality.
-
RESTestBench: A Benchmark for Evaluating the Effectiveness of LLM-Generated REST API Test Cases from NL Requirements
RESTestBench shows that LLM-generated REST API test effectiveness drops when interacting with faulty or mutated code, especially for vague requirements, indicating that high-detail requirements make direct SUT interaction unnecessary.
-
The Semi-Executable Stack: Agentic Software Engineering and the Expanding Scope of SE
Software engineering scope expands beyond executable code to semi-executable artifacts best diagnosed by the new six-ring Semi-Executable Stack model.
-
Do AI Coding Agents Log Like Humans? An Empirical Study
AI agents modify logging less often than humans in 58.4% of repositories but produce higher log density when they change it; explicit logging instructions are rare (4.7%) and ignored 67% of the time, with humans performing 72.5% of post-generation log repairs.
-
Guidelines for Empirical Studies in Software Engineering involving Large Language Models
The paper delivers a taxonomy of seven LLM study types in software engineering along with eight guidelines that separate mandatory requirements from recommended practices to address reproducibility challenges.
-
Rethinking Code Performance Benchmarks for LLMs
Re-evaluating four LLM code-efficiency benchmarks with 30-run statistical testing shows 93.89% of 'performant' implementations are indistinguishable from baselines; a multi-agent test-generation framework reveals hidden significant improvements in ~24% of previously non-significant tasks.
-
Babbling Suppression: Making LLMs Greener One Token at a Time
Babbling Suppression stops LLM code generation upon test passage to reduce token output and energy consumption by up to 65% across Python and Java benchmarks.
-
Efficiency for Experts, Visibility for Newcomers: A Case Study of Label-Code Alignment in Kubernetes
Case study of 18,020 Kubernetes PRs shows label-diff congruence is prevalent and stable, with higher congruence linked to fewer review participants among core developers and more among one-time contributors.
-
Classport: Designing Runtime Dependency Introspection for Java
Classport embeds Maven dependency coordinates into Java class files as runtime annotations and uses a Java agent to report the dependencies actually executed during a run.
-
HiLSVA: Design and Evaluation of a Human-in-the-Loop Agentic System for Scientific Visualization
HiLSVA shows that a human-in-the-loop LLM agent system can help novices and experts complete scientific visualization tasks, while human oversight adds measurable execution time.
-
How Do Developers Use Migration Guides? A Case Study of Log4j
Developers most frequently reference the full Log4j migration guide in pull request descriptions (82.81% of cases) and continue consulting it during post-update maintenance tasks.
-
The Impact of Documentation on Test Engagement in Pull Requests in OSS
Documentation on testing in 160 OSS repositories shows a weak positive correlation (ρ=0.36) with higher test engagement ratios in pull requests, strengthening to moderate in high-activity repos.
-
Towards Predicting Multi-Vulnerability Attack Chains in Software Supply Chains from Software Bill of Materials Graphs
A feasibility study showing SBOM graphs can support component-vulnerability classification and CVE-pair co-exploitation ranking, with acknowledged limits on leakage and generalization.
-
XARP Tools: An Extended Reality Platform for Humans and AI Agents
XARP provides a WebSocket-based remote-procedure system that lets Python code and AI agents control Unity XR clients, with benchmarks and user studies showing faster iteration than conventional XR workflows.
-
Prompt Quality and Pull Request Outcomes: A Stage-Based Empirical Study of LLM-Assisted Development
Specificity and Context predict actionable code generation while Verification predicts adoption and Context predicts integration depth in LLM-assisted PR workflows.
-
Execution-bound advisory automation for agentic AI: a reproducible AIBOM-driven CSAF-VEX framework
Presents a framework for generating cryptographically signed CSAF VEX advisories for agentic AI by binding SBOM/AIBOM artifacts to deterministic environment capture and runtime telemetry.
-
The Impact of Configuring Agentic AI Coding Tools on Build-vs-Buy Decisions: A Study Protocol
The paper outlines a controlled study protocol using staged programming tasks to measure the effects of configuration mechanisms on build-versus-buy decisions in Claude Code and OpenAI Codex.
-
Automated Classification of Human Code Review Comments with Large Language Models
LLMs reach moderate macro-F1 scores of 0.36-0.37 when classifying code review comments into six smells and three useful intents, with one-shot examples helping some models on intent labels.
-
Energy-Aware Computing in the Year 2026
The paper reviews energy-aware computing literature and constructs a taxonomy organized by hardware/software aspects, measurement, optimizations, scheduling, scaling, consolidation, federated learning, and cooling.
- A Comprehensive Evaluation of Code Language Models for Security Patch Detection
- MetaLint: Easy-to-Hard Generalization for Code Linting