REVIEW 8 cited by
Planning with Large Language Models for Code Generation
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Existing large language model-based code generation pipelines typically use beam search or sampling algorithms during the decoding process. Although the programs they generate achieve high token-matching-based scores, they often fail to compile or generate incorrect outputs. The main reason is that conventional Transformer decoding algorithms may not be the best choice for code generation. In this work, we propose a novel Transformer decoding algorithm, Planning-Guided Transformer Decoding (PG-TD), that uses a planning algorithm to do lookahead search and guide the Transformer to generate better programs. Specifically, instead of simply optimizing the likelihood of the generated sequences, the Transformer makes use of a planner to generate candidate programs and test them on public test cases. The Transformer can therefore make more informed decisions and generate tokens that will eventually lead to higher-quality programs. We also design a mechanism that shares information between the Transformer and the planner to make our algorithm computationally efficient. We empirically evaluate our framework with several large language models as backbones on public coding challenge benchmarks, showing that 1) it can generate programs that consistently achieve higher performance compared with competing baseline methods; 2) it enables controllable code generation, such as concise codes and highly-commented codes by optimizing modified objective.
Forward citations
Cited by 8 Pith papers
-
Solver-Aware Decompositions for Programming-by-Example: When Dividing Requires Knowing how to Conquer
Solver-aware training of a PBE decomposer with a frozen synthesizer's loss outperforms supervised imitation of ground-truth subgoals, solving tasks that a ground-truth decomposition oracle fails.
-
MOCHA: Are Code Language Models Robust Against Multi-Turn Malicious Coding Prompts?
MOCHA is a benchmark of 10.5K malicious coding prompts, including multi-turn decomposition attacks, showing code LLMs reject these incremental attacks at much lower rates and that fine-tuning on the benchmark improves...
-
LogiDroid: Individual Functional Test Generation via Business Logic Extraction and Adaptation
LogiDroid generates functional Android test cases with verification assertions by retrieving similar test cases, fusing their business logic, and adapting it to the target app's real-time GUI state.
-
Concentration bounds on response-based vector embeddings of black-box generative models
For bounded-variance response distributions, the DKPS embedding error is O_P((n^3/r)^{1/2-delta}) when r grows faster than n^3.
-
It's Not That Simple. An Analysis of Simple Test-Time Scaling
The apparent scaling behavior of the s1 model is mostly an artifact of truncating long outputs, while appending 'Wait' does not reliably improve answers.
-
Breaking the Myth: Can Small Models Infer Postconditions Too?
Fine-tuning Qwen2.5-Coder-7B on 1.5K reasoning examples yields postcondition generation close to GPT-4o and better than 32B open models on Defects4J.
-
MGC: A Compiler Framework Exploiting Compositional Blindness in Aligned LLMs for Malware Generation
MGC, a two-stage compiler framework, generates functional malware by decomposing malicious intents into benign-appearing MDIR components that strong aligned LLMs will implement, bypassing safety alignment.
-
BLUEX Revisited: Enhancing Benchmark Coverage with Automatic Captioning
An updated BLUEX benchmark with 1,422 questions and GPT-4o-generated captions that make image-based questions usable by text-only LLMs.
Discussion (0). Sign in to comment.