REVIEW 8 cited by
DocPrompting: Generating Code by Retrieving the Docs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
DocPrompting: Generating Code by Retrieving the Docs
read the original abstract
Publicly available source-code libraries are continuously growing and changing. This makes it impossible for models of code to keep current with all available APIs by simply training these models on existing code repositories. Thus, existing models inherently cannot generalize to using unseen functions and libraries, because these would never appear in the training data. In contrast, when human programmers use functions and libraries for the first time, they frequently refer to textual resources such as code manuals and documentation, to explore and understand the available functionality. Inspired by this observation, we introduce DocPrompting: a natural-language-to-code generation approach that explicitly leverages documentation by (1) retrieving the relevant documentation pieces given an NL intent, and (2) generating code based on the NL intent and the retrieved documentation. DocPrompting is general: it can be applied to any programming language and is agnostic to the underlying neural model. We demonstrate that DocPrompting consistently improves NL-to-code models: DocPrompting improves strong base models such as CodeT5 by 2.85% in pass@1 (52% relative gain) and 4.39% in pass@10 (30% relative gain) in execution-based evaluation on the popular Python CoNaLa benchmark; on a new Bash dataset tldr, DocPrompting improves CodeT5 and GPT-Neo1.3B by up to absolute 6.9% exact match.
Forward citations
Cited by 8 Pith papers
-
RepoReasoner: Evaluating Repository-Level Code Reasoning Ability of Long-Context Language Models
RepoReasoner is a repository-level code-reasoning benchmark with output-prediction and call-chain tasks; the best LLM reaches only 69.1% Pass@1 even with oracle context, with low recall in dependency tracing.
-
BIM Information Extraction Through LLM-based Adaptive Exploration
LLM adaptive exploration via runtime code execution outperforms static query generation for information extraction from heterogeneous BIM models on the new ifc-bench v2 benchmark.
-
PHITSBench: an execution-scored benchmark for AI-assisted PHITS radiation-transport input generation using natural language
A structured PHITS knowledge catalog lifts GPT-5.4 from 0% to 57% success generating full radiation-transport simulations; agentic repair reaches 66–73%.
-
Co-Located Tests, Better AI Code: How Test Syntax Structure Affects Foundation Model Code Generation
Co-locating tests with implementation code yields substantially higher preservation and correctness in foundation-model-generated programs than separated test syntax.
-
Beyond "What to Retrieve": Uncertainty in Retrieval-Augmented Code Generation
Uncertainty-aware multi-source retrieval improves GPT repository-level code selection over plain RAG but matches verification-and-repair alone and is backend- and context-dependent.
-
Not All RAGs Are Created Equal: A Component-Wise Empirical Study for Software Engineering Tasks
Retriever-side choices, particularly the retrieval algorithm, exert more influence on RAG performance than generator selection across code generation, summarization, and repair tasks.
-
ALGOGEN: Tool-Generated Verifiable Traces for Reliable Algorithm Visualization
ALGOGEN improves LLM-generated algorithm visualizations by splitting simulation into traceable JSON outputs via Visualization Trace Algebra and using Rendering Style Language for reliable rendering, raising success ra...
-
VeriInteresting: An Empirical Study of Model Prompt Interactions in Verilog Code Generation
Empirical study identifies patterns in how model classes respond to structured prompts, optimization, and other techniques across two Verilog benchmarks.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.