LLMs frequently specify library versions with known CVEs in generated code (36-56% of tasks), show low compatibility (20-63%), and converge on the same risky versions across models.
Canonical reference
Title resolution pending
Canonical reference. 71% of citing Pith papers cite this work as background.
37
Pith papers citing it
587
external citations · Crossref
Background
71% of classified citations
citation-role summary
background 6
dataset 1
citation-polarity summary
representative citing papers
Incomplete constrainers in constrained decoding push LLMs into low-probability program regions, making unconstrained decoding outperform constrained decoding on functional correctness across seven models and three benchmarks.
A new six-dimension process taxonomy for AI software development frameworks shows convergence on artifact persistence and human oversight but reveals that no framework covers all dimensions strongly, indicating a depth-portability trade-off.
A matched-pair protocol and Accurate Differentiation Rate metric reveal that conventional LLM accuracy on SAT problems is often inflated by over-predicting satisfiability, while cross-representation agreement exceeds 80 percent for most models.
Reversa is a reverse documentation engineering framework that deploys a multi-agent pipeline to extract implicit rules from legacy software and produce traceable specifications with confidence scores and explicit gaps for human review.
Event-B Agent is an LLM agent that synthesizes, refines, and repairs Event-B formal models from natural language requirements via iterative verification feedback loops.
InteractWeb-Bench shows that frontier multimodal AI agents remain trapped in blind execution when generating websites from perturbed, low-quality non-expert instructions.
A systematic review of 50 studies identifies 69 LLM-assisted tasks in empirical software engineering, concentrated in data processing and analysis with gaps in human-centered integration and reproducibility reporting.
Software engineering scope expands beyond executable code to semi-executable artifacts best diagnosed by the new six-ring Semi-Executable Stack model.
A survey of 457 SE researchers finds widespread GenAI use concentrated in writing and ideation, with productivity gains but persistent concerns over accuracy, bias, and the need for clearer governance rules.
A large-scale mining study identifies and characterizes 2,502 verified DDD repositories on GitHub, revealing a 2017 adoption inflection point, C#/TypeScript language dominance, and sustained professional-grade engineering activity.
Plainbook makes data-science notebooks natural-language-first by preserving cell descriptions, generating code via AI, enforcing linear execution via a checkpointing kernel, and adding value-centered cell and global tests.
Auditing three performance-optimization benchmarks shows reference-patch validity collapses under cross-machine replay, rankings depend on scoring rules, and 384 of 450 replay-valid tasks are already solved by at least one public submission.
IntentTester migrates tests across libraries using TDL abstraction and multi-agent LLM synthesis, achieving 85% correctness and 74% effectiveness versus 51% and 43% for baselines on nine projects in JSON, HTML, and Time domains.
CAPRA is a multi-agent LLM system with evidence anchoring and consistency checking that analyzes software architecture deliverables and meets 88.8% of an eight-criterion evaluation on 10 student reports.
SWE-Mutation benchmark shows current LLMs achieve low verification (10.20%) and detection (36.15%) rates on 2,636 mutated variants, exposing weaknesses in generating reliable test suites.
A structured JSON intermediate representation for LLM-generated static analysis queries outperforms both direct generation and agentic tool use, with gains of 15-25 percentage points on large models.
A neuro-symbolic pipeline (LLM function summaries + Z3 path checks) injected into CodeQL/Infer finds 54 memory leaks on 3.6M LOC of real C/C++ software, 53 confirmed, versus 19 and 3 for the baselines.
LLM approaches ExArch and ArTEMiS reach F1 scores of 0.86 and 0.81 for architecture entity recognition and traceability, matching or approaching baselines that require manual models.
A systematic mapping study of 45 LLM-based RE papers identifies and characterizes 62 public datasets, revealing imbalances in open-science practices, elicitation support, and socio-technical diversity.
ATM is a CID-brokered governance framework that maps write intents to semantic atoms for pre-admission control, validation, and neutral-steward application in single-domain multi-agent code synthesis.
Eye-tracking experiment finds that labeling code as LLM-generated increases fixation time without changing review thoroughness, with reviewers adapting criteria or using the prompt.
CodeXHug is a curated dataset of 7,325 HuggingFace PTMs and 20,545 Python files from GitHub, demonstrated via statistical analysis and clustering to extract code usage patterns.
Mixed-methods study of 27 developers characterizes five Copilot chat interaction modes and ten needs linked to problem-solving styles and experience levels.
citing papers explorer
-
Correct Code, Vulnerable Dependencies: A Large Scale Measurement Study of LLM-Specified Library Versions
LLMs frequently specify library versions with known CVEs in generated code (36-56% of tasks), show low compatibility (20-63%), and converge on the same risky versions across models.
-
The Alignment Problem in Constrained Code Generation
Incomplete constrainers in constrained decoding push LLMs into low-probability program regions, making unconstrained decoding outperform constrained decoding on functional correctness across seven models and three benchmarks.
-
From Prompt to Process: a Process Taxonomy and Comparative Assessment of Frameworks Supporting AI Software Development Agents
A new six-dimension process taxonomy for AI software development frameworks shows convergence on artifact persistence and human oversight but reveals that no framework covers all dimensions strongly, indicating a depth-portability trade-off.
-
Satisfiability Solving with LLMs: A Matched-Pair Evaluation of Reasoning Capability
A matched-pair protocol and Accurate Differentiation Rate metric reveal that conventional LLM accuracy on SAT problems is often inflated by over-predicting satisfiability, while cross-representation agreement exceeds 80 percent for most models.
-
Reversa: A Reverse Documentation Engineering Framework for Converting Legacy Software into Operational Specifications for AI Agents
Reversa is a reverse documentation engineering framework that deploys a multi-agent pipeline to extract implicit rules from legacy software and produce traceable specifications with confidence scores and explicit gaps for human review.
-
Event-B Agent: Towards LLM Agent for Formal Model Synthesis and Repair
Event-B Agent is an LLM agent that synthesizes, refines, and repairs Event-B formal models from natural language requirements via iterative verification feedback loops.
-
InteractWeb-Bench: Can Multimodal Agent Escape Blind Execution in Interactive Website Generation?
InteractWeb-Bench shows that frontier multimodal AI agents remain trapped in blind execution when generating websites from perturbed, low-quality non-expert instructions.
-
LLM-Assisted Empirical Software Engineering: Systematic Literature Review and Research Agenda
A systematic review of 50 studies identifies 69 LLM-assisted tasks in empirical software engineering, concentrated in data processing and analysis with gaps in human-centered integration and reproducibility reporting.
-
The Semi-Executable Stack: Agentic Software Engineering and the Expanding Scope of SE
Software engineering scope expands beyond executable code to semi-executable artifacts best diagnosed by the new six-ring Semi-Executable Stack model.
-
Taking a Pulse on How Generative AI is Reshaping the Software Engineering Research Landscape
A survey of 457 SE researchers finds widespread GenAI use concentrated in writing and ideation, with productivity gains but persistent concerns over accuracy, bias, and the need for clearer governance rules.
-
Domain-Driven Design in Practice: A Large-Scale Empirical Characterisation of the Open-Source Ecosystem
A large-scale mining study identifies and characterizes 2,502 verified DDD repositories on GitHub, revealing a 2017 adoption inflection point, C#/TypeScript language dominance, and sustained professional-grade engineering activity.
-
Plainbook: Data Science, in Plain Language
Plainbook makes data-science notebooks natural-language-first by preserving cell descriptions, generating code via AI, enforcing linear execution via a checkpointing kernel, and adding value-centered cell and global tests.
-
Are Performance-Optimization Benchmarks Reliably Measuring Coding Agents?
Auditing three performance-optimization benchmarks shows reference-patch validity collapses under cross-machine replay, rankings depend on scoring rules, and 384 of 450 replay-valid tasks are already solved by at least one public submission.
-
IntentTester: Intent-Driven Multi-agent Framework for Cross-Library Test Migration
IntentTester migrates tests across libraries using TDL abstraction and multi-agent LLM synthesis, achieving 85% correctness and 74% effectiveness versus 51% and 43% for baselines on nine projects in JSON, HTML, and Time domains.
-
CAPRA: Scaling Feedback on Software Architecture Deliverables with a Multi-Agent LLM System
CAPRA is a multi-agent LLM system with evidence anchoring and consistency checking that analyzes software architecture deliverables and meets 88.8% of an eight-criterion evaluation on 10 student reports.
-
SWE-Mutation: Can LLMs Generate Reliable Test Suites in Software Engineering?
SWE-Mutation benchmark shows current LLMs achieve low verification (10.20%) and detection (36.15%) rates on 2,636 mutated variants, exposing weaknesses in generating reliable test suites.
-
Less Is More: Measuring How LLM Involvement affects Chatbot Accuracy in Static Analysis
A structured JSON intermediate representation for LLM-generated static analysis queries outperforms both direct generation and agentic tool use, with gains of 15-25 percentage points on large models.
-
Finding Memory Leaks in C/C++ Programs via Neuro-Symbolic Augmented Static Analysis
A neuro-symbolic pipeline (LLM function summaries + Z3 path checks) injected into CodeQL/Infer finds 54 memory leaks on 3.6M LOC of real C/C++ software, 53 confirmed, versus 19 and 3 for the baselines.
-
Who's Who? LLM-assisted Software Traceability with Architecture Entity Recognition
LLM approaches ExArch and ArTEMiS reach F1 scores of 0.86 and 0.81 for architecture entity recognition and traceability, matching or approaching baselines that require manual models.
-
Characterizing Datasets for LLM-based Requirements Engineering: A Systematic Mapping Study
A systematic mapping study of 45 LLM-based RE papers identifies and characterizes 62 public datasets, revealing imbalances in open-science practices, elicitation support, and socio-technical diversity.
-
ATM: CID-Brokered Pre-Write Admission for Multi-Agent Code Co-Synthesis
ATM is a CID-brokered governance framework that maps write intents to semantic atoms for pre-admission control, validation, and neutral-steward application in single-domain multi-agent code synthesis.
-
Same Scrutiny, More Time: Eye Tracking Insights into Reviewing LLM-Labelled Code
Eye-tracking experiment finds that labeling code as LLM-generated increases fixation time without changing review thoroughness, with reviewers adapting criteria or using the prompt.
-
Generate with CodeXHug: A Dataset to Enhance Model Cards with Code Usage Patterns
CodeXHug is a curated dataset of 7,325 HuggingFace PTMs and 20,545 Python files from GitHub, demonstrated via statistical analysis and clustering to extract code usage patterns.
-
No Two Developers Think Alike: How Problem-Solving Styles and Experience Shape Needs in Conversational Interaction with Copilot
Mixed-methods study of 27 developers characterizes five Copilot chat interaction modes and ten needs linked to problem-solving styles and experience levels.
-
Can AI be Easy? Lessons Learned from the EZR.py Toolkit
EZR.py shows that a compact, readable Python toolkit can match or exceed state-of-the-art tools like SHAP, LIME, SMAC3, and FASTREAD on over 120 tabular SE tasks while running 500 times faster and using far less labeled data.
-
Domain-Adaptable Reinforcement Learning for Code Generation with Dense Rewards
A PPO-based RL framework with execution-aware dense rewards and token-level mapping improves pass@1 by 19% on MBPP and reduces execution failures by 51% on RoboEval for LLM code generation.
-
Causal Software Engineering: A Vision and Roadmap
Causal Software Engineering is proposed as a paradigm that applies causal models and counterfactual reasoning to inform high-stakes decisions throughout software development and operations.
-
AgentReputation: A Decentralized Agentic AI Reputation Framework
AgentReputation proposes separating AI agent task execution, reputation management, and secure record-keeping into distinct layers, with context-specific reputation cards and a risk-based policy engine to handle verification in decentralized settings.
-
Exploring Creativity in Human-Human-LLM Collaborative Software Design
Creativity in human-LLM collaborative software design emerges primarily from human traits and interactions, with LLMs providing supplementary novel ideas but occasionally hindering progress.
-
The Role of LLMs in Collaborative Software Design
Exploratory lab study finds shared LLM use builds shared understanding in design teams while parallel use risks context drift, with professionals reflecting on outputs for insights but sometimes anchoring early.
-
Walking the Tightrope of LLMs for Software Development: A Practitioners' Perspective
Qualitative interview study with 22 practitioners identifies multi-level benefits, challenges, and mitigation strategies for using LLMs in software development.
-
Prompt Quality and Pull Request Outcomes: A Stage-Based Empirical Study of LLM-Assisted Development
Specificity and Context predict actionable code generation while Verification predicts adoption and Context predicts integration depth in LLM-assisted PR workflows.
-
Recommendations for Efficient and Responsible LLM Adoption within Industrial Software Development
A multi-case study plus survey produces seven actionable recommendations for efficient and responsible LLM use in industrial software engineering.
-
Combining Static Code Analysis and Large Language Models Improves Correctness and Performance of Algorithm Recognition
Hybrid LLM plus static analysis for algorithm recognition in code cuts required model calls by 72-97% and lifts F1-scores by as much as 12 points.
-
Personalizing LLM-Based Conversational Programming Assistants
The paper describes ongoing efforts to characterize developer diversity in cognition and context and to use personalization to make LLM-based conversational programming assistants more inclusive.
- What Breaks When LLMs Code? Characterizing Operational Safety Failures of Agentic Code Assistants
- AgentModernize: Preserving Business Logic in Legacy Modernization with Multi-Agent LLMs and Behavioral Specification Graphs