REVIEW 21 cited by
Large Language Models for Code Generation: A Comprehensive Survey of Challenges, Techniques, Evaluation, and Applications
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large Language Models (LLMs) have demonstrated their remarkable capabilities in numerous fields. This survey focuses on how LLMs empower users, regardless of their technical background, to use human languages to automatically generate executable code. We begin with understanding LLMs' limitations and challenges in automated code generation. Subsequently, we review various fine-tuning techniques designed to enhance both the performance and adaptability of LLMs in code generation tasks. We then review the existing metrics and benchmarks for evaluations to assess model performance based on fine-tuning techniques. Finally, we explore the applications of LLMs (e.g. CodeLlama, GitHub Copilot, ToolGen) in code generation tasks to illustrate their roles and functionalities. This survey provides a comprehensive overview of LLMs for code generation, helps researchers in diverse fields better understand the current state-of-the-art technologies, and offers the potential of effectively leveraging LLMs for code generation tasks.
Forward citations
Cited by 21 Pith papers
-
ACE: Self-Evolving LLM Coding Framework via Adversarial Unit Test Generation and Preference Optimization
ACE uses a solver-adversary loop with adversarial unit test generation and execution-based preference optimization to enable self-evolving LLM code generation, reporting 3-7% pass@1 gains over solver-verifier baseline...
-
MultiMat: Multimodal Program Synthesis for Procedural Materials using Large Multimodal Models
MultiMat shows multimodal large models plus constrained search produce higher-quality procedural material graphs than text-only baselines on a new production dataset.
-
TraceDev: A Traceability-Driven Multi-agent Framework for Requirement-to-Code Development
A traceability-graph-driven multi-agent framework improves LLM-based repository-level code generation from use cases, reaching 53-57% functional success on ETOUR/SMOS versus 13-25% for ChatDev and MetaGPT.
-
LLM-based Mockless Unit Test Generation for Java
MocklessTester generates mockless Java unit tests using context mining and multi-level constraint fixing, improving coverage and mutation scores over baselines on Defects4J and Deps4J.
-
Enhancing Reliability in LLM-Based Secure Code Generation
MA-CoT prompting reduces security findings in LLM-generated code by 57.6% on a 200-task dataset and 94.5% on LLMSecEval across C, Java, and Python, outperforming vanilla, zero-shot, and standard CoT strategies.
-
VeriContest: A Competitive-Programming Benchmark for Verifiable Code Generation
VeriContest supplies 946 problems with specs, code, proofs, and tests to benchmark verifiable code generation in Rust/Verus, showing models reach 92% on code but only 5% end-to-end on full verifiable synthesis.
-
Bridging Generation and Training: A Systematic Review of Quality Issues in LLMs for Code
A review of 114 studies creates taxonomies for code and data quality issues, formalizes 18 propagation mechanisms from training data defects to LLM-generated code defects, and synthesizes detection and mitigation techniques.
-
Token-Level LLM Collaboration via FusionRoute
FusionRoute augments token-level expert routing with a trainable complementary logit generator to expand the policy class and recover optimal decoding under mild conditions, outperforming prior collaboration and mergi...
-
A Rule-Aware Prompt Framework for Structured Numeric Reasoning in Cyber-Physical Systems
A rule-aware modular prompt framework enables LLMs to perform structured numeric reasoning on power grid data by separating rules from normalized deviations, improving anomaly detection consistency and reducing token ...
-
Can Vibe Coding Beat Graduate CS Students? An LLM vs. Human Coding Tournament on Market-driven Strategic Planning
Student-coded agents from a 2020 logistics course beat 40 LLM-coded agents in a bidding-and-delivery tournament, with 33 of 40 AI agents losing to a simple baseline.
-
MermaidSeqBench: An Evaluation Benchmark for NL-to-Mermaid Sequence Diagram Generation
MermaidSeqBench is a new human-verified benchmark for evaluating LLMs on natural language to Mermaid sequence diagram generation, revealing significant capability gaps across models.
-
Testing chatbots on the creation of encoders for audio conditioned image generation
All chatbot-designed audio encoders failed to align with CLIP text embeddings and produced incoherent images, while showing a surprising architectural similarity across chatbots.
-
Simulation Code Generation for Fluid Systems using Large Language Models: Benchmarking Models and Prompting Strategies
Across 10 LLMs and 5 prompting strategies, curated context yields the best syntactic quality, yet simulation fidelity stays near zero for nearly all configurations.
-
SiriusHelper: An LLM Agent-Based Operations Assistant for Big Data Platforms
SiriusHelper deploys an LLM agent with intent routing, DeepSearch multi-hop retrieval, and automated SOP distillation to outperform alternatives and reduce ticket volume by 20.8% on Tencent's big data platform.
-
ACE: Self-Evolving LLM Coding Framework via Adversarial Unit Test Generation and Preference Optimization
ACE introduces a solver-adversary loop where an LLM generates both candidate programs and adversarial tests, using execution outcomes for preference optimization to achieve 3-7% pass@1 gains on code benchmarks without...
-
RAG Strategies for Natural Language-Based SQL Query and REST API Call Generation
In a 631-case enterprise benchmark, iterative retrieval (CoRAG) achieved 10.29% exact match vs 7.45% for standard RAG on combined SQL/API generation with hybrid documentation, while no-RAG gives 0% exact match.
-
AdaDec: A Uncertainty-Guided Lookahead Decoding Framework for LLM-Based Code Generation
AdaDec improves Pass@1 accuracy of LLM code generation by up to 20.9% over greedy decoding by triggering lookahead reranking only at high-uncertainty steps on HumanEval+, MBPP+, and DevEval.
-
Exp2VLA: Enabling Vision-Language-Action for Drone Navigation from Expert Demonstrations
Expert demonstrations distilled into LeRobot-format data let fine-tuned mid-scale VLAs navigate multi-object drone scenes from language, with partial zero-shot color-shape generalization in simulation and SITL.
-
Architectures of Error: A Philosophical Inquiry into AI and Human Code Generation
Develops a conceptual distinction between human-cognitive and artificial-stochastic error architectures in code generation, drawing on Dennett, Rescher, and Floridi to explore implications for AI-human collaboration.
-
Sustainable Code Generation Using Large Language Models: A Systematic Literature Review
A systematic review finds research on the sustainability of LLM-generated code to be limited, fragmented, and without accepted frameworks for measurement or benchmarking.
-
LLM-Based Multi-Agent Systems for Code Generation: A Multi-Vocal Literature Review
A review of 114 studies classifies motivations into nine categories, analyzes common models and benchmarks, synthesizes challenges into six categories with 26 subcategories and solutions, and identifies six future res...
Discussion (0). Sign in to comment.