REVIEW 9 cited by
Debug like a Human: A Large Language Model Debugger via Verifying Runtime Execution Step-by-step
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Large language models (LLMs) are leading significant progress in code generation. Beyond one-pass code generation, recent works further integrate unit tests and program verifiers into LLMs to iteratively refine the generated programs. However, these works consider the generated programs as an indivisible entity, which falls short for LLMs in debugging the programs, especially when the programs contain complex logic flows and data operations. In contrast, when human developers debug programs, they typically set breakpoints and selectively examine runtime execution information. The execution flow and the intermediate variables play a crucial role in the debugging process, yet they are underutilized in the existing literature on code generation. In this study, we introduce Large Language Model Debugger (LDB), a novel debugging framework that enables LLMs to refine their generated programs with the runtime execution information. Specifically, LDB segments the programs into basic blocks and tracks the values of intermediate variables after each block throughout the runtime execution. This allows LLMs to concentrate on simpler code units within the overall execution flow, verify their correctness against the task description block by block, and efficiently pinpoint any potential errors. Experiments demonstrate that LDB consistently enhances the baseline performance by up to 9.8% across the HumanEval, MBPP, and TransCoder benchmarks, archiving new state-of-the-art performance in code debugging for various LLM selections.
Forward citations
Cited by 9 Pith papers
-
ReCode: Improving LLM-based Code Repair with Fine-Grained Retrieval-Augmented Generation
ReCode improves LLM-based code repair by combining algorithm-type prediction with dual-encoder retrieval of authentic bug-fix pairs, and introduces RACodeBench for evaluation.
-
HLSDebugger: Identification and Correction of Logic Bugs in HLS Code with LLM Solutions
HLSDebugger fixes logic bugs in HLS code with 37.6% accuracy, outperforming GPT-4 by over 3x, using an encoder-decoder model trained on a new 300K-sample dataset.
-
CodeReasoner: Enhancing the Code Reasoning Ability with Reinforcement Learning
CodeReasoner combines a concise execution-focused dataset, instruction tuning, and GRPO RL to make 7B/14B models match or beat GPT-4o on code reasoning benchmarks.
-
Measuring and Augmenting Large Language Models for Solving Capture-the-Flag Challenges
A benchmark and agent for CTF solving, but the agent's retrieval database appears to contain the answers to the test challenges, undermining the reported improvements.
-
Detection Time Distribution Predicted Using Absorbing Boundary Conditions and Imaginary Potentials
Three inequivalent quantum detection-time proposals yield waveguide arrival distributions showing partial reflection, spin-orientation independence (for tested parameters), and width dependence when spin couples to th...
-
Let's Revise Step-by-Step: A Unified Local Search Framework for Code Generation with LLMs
The abstract claims a new local search framework for code generation, but the manuscript body is a different mathematics paper.
-
Running in CIRCLE? A Simple Benchmark for LLM Code Interpreter Security
CIRCLE is a 1,260-prompt benchmark that measures how often commercial LLM code interpreters refuse, execute, or time out on resource-exhaustion tasks, revealing large and inconsistent safety gaps.
-
Xolver: Multi-Agent Reasoning with Holistic Experience Learning Just Like an Olympiad Team
A training-free multi-agent framework with episodic and shared memory reports new best results on GSM8K, AIME 2024/2025, Math-500, and LiveCodeBench.
-
Enhancing LLM-Based Code Generation with Complexity Metrics: A Feedback-Driven Approach
Feeding LLMs their own code's complexity metrics as feedback modestly improves Pass@1 on some benchmarks, especially for weaker models like GPT-3.5 Turbo.
Discussion (0). Sign in to comment.