REVIEW 16 cited by
Large Language Models for Software Engineering: Survey and Open Problems
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
This paper provides a survey of the emerging area of Large Language Models (LLMs) for Software Engineering (SE). It also sets out open research challenges for the application of LLMs to technical problems faced by software engineers. LLMs' emergent properties bring novelty and creativity with applications right across the spectrum of Software Engineering activities including coding, design, requirements, repair, refactoring, performance improvement, documentation and analytics. However, these very same emergent properties also pose significant technical challenges; we need techniques that can reliably weed out incorrect solutions, such as hallucinations. Our survey reveals the pivotal role that hybrid techniques (traditional SE plus LLMs) have to play in the development and deployment of reliable, efficient and effective LLM-based SE.
Forward citations
Cited by 16 Pith papers
-
Combining TSL and LLM to Automate REST API Testing: A Comparative Study
RestTSLLM combines Test Specification Language with few-shot prompting to generate REST API integration tests, and a comparative study finds Claude 3.5 Sonnet the best among eight LLMs.
-
Retrieval-Augmented Generation for Service Discovery: Chunking Strategies and Benchmarking
Endpoint-based chunking outperforms naive splitting for RAG-based endpoint discovery, and an LLM agent raises precision at the cost of recall, evaluated on a new multi-domain benchmark and RestBench.
-
Automatically Generating Rules of Malicious Software Packages via Large Language Model
RuleLLM automatically generates YARA and Semgrep rules from malware metadata and code, reporting 85.2% precision and 91.8% recall on its 1,633-package corpus.
-
Fault Localization via Fine-tuning Large Language Models with Mutation Generated Stack Traces
Fine-tuning LLMs on mutation-generated crash stack traces localizes faults in SAP HANA, SQLite, and DuckDB with 63 to 74 percent accuracy, beating weak baselines.
-
Practitioners' Expectations on Log Anomaly Detection
Practitioners want log anomaly detectors that are accurate, fast, explainable, and customizable, but current research rarely delivers on the last three.
-
LLMPirate: LLMs for Black-box Hardware IP Piracy
LLMPirate uses eight LLMs to rewrite Verilog netlists into functionally equivalent circuits that evade GNN4IP, MOSS, Jplag, and SIM on most tested designs.
-
Akashic: A Low-Overhead LLM Inference Service with MemAttention
Akashic’s MemAttention plus locality-aware placement improves agent task accuracy by up to 10.2 points and throughput by up to 1.21× over prior memory systems across four long-horizon workloads.
-
Quantize with Confidence? An Empirical Study of Quantization for Code Generation
4-bit quantization of code LLMs mostly preserves pass@1, but AQLM matches or beats full precision while QuIP# degrades most on complex prompts; sensitivity is model-dependent.
-
Development of Automated Software Design Document Review Methods Using Large Language Models
Converting tabular design documents into header-aware Markdown or JSON formats lets GPT models catch cross-document inconsistencies with recall up to 0.96 on short documents, but performance collapses beyond 5000 characters.
-
Generating Proto-Personas through Prompt Engineering: A Case Study on Efficiency, Effectiveness and Empathy
A prompt-engineering approach to proto-persona generation, evaluated with 19 practitioners in a real Lean Inception, reduced creation time to about six minutes and was well accepted, but affective and behavioral empat...
-
Mobile Application Review Summarization using Chain of Density Prompting
Adapting the Chain of Density prompt to define entities as app features yields denser and more readable summaries of mobile app reviews than the original prompt, vanilla prompting, or extractive baselines.
-
Advanced System Integration: Analyzing OpenAPI Chunking for Retrieval-Augmented Generation
An empirical study showing that LLM-generated endpoint summaries and an agent that retrieves details on demand improve OpenAPI endpoint discovery in a RAG system, at the cost of lower recall.
-
CODECLEANER: Elevating Standards with A Robust Data Contamination Mitigation Toolkit
A toolkit of 11 code refactoring operators reduces n-gram overlap with training corpora by up to 65 percentage points, though this drop is partly by construction and is not tied to downstream task performance.
-
From Inductive to Deductive: LLMs-Based Qualitative Data Analysis in Requirements Engineering
GPT-4 labels software requirements with substantial agreement to human analysts (Cohen's Kappa up to 0.738) when given detailed few-shot prompts, while zero-shot performance is only moderate.
-
Technical Challenges in Maintaining Tax Prep Software with Large Language Models
A combination of CodeBERTScore and majority voting ranks LLM-generated tax code updates better than either metric alone, but results are preliminary and limited by small samples.
-
The Road to Artificial SuperIntelligence: A Comprehensive Survey of Superalignment
A survey of scalable oversight for superalignment, reviewing weak-to-strong generalization, debate, RLAIF, and sandwiching, and concluding that current methods are not yet sufficient for superintelligent AI.
Discussion (0). Continue with ORCID to comment.