Pith. sign in

REVIEW 3 major objections 2 minor 2 cited by

ReLoc unifies step-by-step code revision into a local search loop that the paper claims beats tree search and prior improvement methods on code generation tasks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

The abstract claims a new local search framework for code generation, but the manuscript body is a different mathematics paper.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection The abstract describes a plausible local-search framework for code revision, but the supplied full text is a different commutative algebra paper, leaving every experimental claim unverifiable. the 3 major comments →

arxiv 2508.07434 v1 pith:XX2V66XT submitted 2025-08-10 cs.CL

Let's Revise Step-by-Step: A Unified Local Search Framework for Code Generation with LLMs

Zhiyi Lyu , Jianguo Huang , Yanchen Deng , Steven Hoi , Bo An This is my paper
classification cs.CL
keywords code generationlarge language modelsinference-time scalinglocal searchcode revisionhill climbinggenetic algorithmreward model
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes ReLoc, a unified local search framework for code generation with large language models, and claims it outperforms both construction-based tree-search methods and prior improvement-based methods. The central move is to treat code generation as an incremental revision process: draft an initial program, generate nearby variants, score them, and keep the best, repeating until the compute budget runs out. This loop can be instantiated as Hill Climbing or a Genetic Algorithm, making it a single recipe for what were previously separate search strategies. The paper further argues that a dedicated revision reward model, which judges a candidate by how far it has moved from earlier versions, supplies the fine-grained signal that guides the search toward promising candidates. If correct, ReLoc would make inference-time scaling for code generation more token-efficient and anytime, since it returns the current best candidate whenever the budget ends.

Core claim

The paper claims that code revision can be organized as local search over program space, and that this organization is superior to both growing a tree of partial solutions and to earlier improvement-only methods. ReLoc's four components—initial code drafting, neighborhood code generation, candidate evaluation, and incumbent code updating—are sufficient to implement classic local search algorithms, and different decision rules instantiate Hill Climbing or a Genetic Algorithm. The enabling mechanism is a revision reward model trained to rank candidates by their revision distance from the current incumbent, providing a dense preference signal that generic code-quality scores lack. On diverse co

What carries the argument

The four-step local search loop—draft an initial program, generate neighboring variants, evaluate candidates with a reward model, and update the incumbent—instantiable as Hill Climbing or a Genetic Algorithm. The revision reward model, which scores a candidate by its revision distance from earlier versions, is the component that converts this loop into a well-guided, anytime search process.

Load-bearing premise

ReLoc's performance depends on the revision reward model treating revision distance as a faithful proxy for code-quality improvement; if a candidate can move far while getting worse, the search loses its signal.

What would settle it

Ablate the revision-distance reward model by replacing it with a generic code-quality scorer or random neighbor selection while keeping the same local search loop; if final pass rates on standard benchmarks like HumanEval or MBPP do not drop, the paper's central mechanism is not the source of its reported gains.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • ReLoc provides a unified recipe that covers multiple local search algorithms, so insights about one algorithm's exploration-exploitation tradeoffs transfer to code revision.
  • The anytime property lets users trade token budget for solution quality at runtime, unlike construction-based tree search which must commit to growing a full tree.
  • The revision reward model suggests that relative change between candidates is a more informative training signal than absolute code quality, a claim the paper bases on its experimental results.
  • Outperforming both construction-based and improvement-based methods implies that step-by-step revision is the more scalable direction for inference-time scaling of code LLMs.
  • Because the loop treats revision as a generic operation, the framework is task-agnostic across code generation benchmarks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same revision-distance reward could transfer to other iterative-refinement domains—mathematical proof generation, text editing, or program repair—where the path of small corrections matters as much as the final artifact.
  • ReLoc's anytime property suggests it could be combined with tree-search methods: expand a sparse tree for breadth, then run ReLoc's local steps within promising branches to spend tokens where they pay off.
  • A testable extension is to instantiate the framework with other local search metaheuristics, such as simulated annealing or tabu search, to see whether the reported gains come from the loop's structure or from the specific Hill Climbing and Genetic Algorithm choices.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The submission's abstract describes ReLoc, a unified local search framework for LLM-based code generation. It proposes four algorithmic components—initial code drafting, neighborhood code generation, candidate evaluation, and incumbent code updating—that can be instantiated as Hill Climbing or Genetic Algorithm, together with a specialized revision reward model that scores code quality by revision distance. The abstract claims extensive experimental results showing superior performance over construction-based tree search and state-of-the-art improvement-based methods. However, the supplied full text is not the described paper: it is arXiv:2508.07437v1 [math.AC], a commutative algebra paper on joint reductions and mixed Buchsbaum-Rim multiplicities. Consequently, none of the methodological details, experimental setups, baselines, or results for ReLoc appear anywhere in the manuscript.

Significance. If the claimed results held, ReLoc would be a potentially useful contribution to inference-time scaling for code generation, and the modular framing through local-search instantiations is conceptually clear. However, significance cannot be assessed from the submitted material: the manuscript contains no description of the proposed framework beyond the abstract, no experimental design, no datasets, no baselines, and no numerical results. The central empirical claim is therefore entirely unsupported.

major comments (3)
  1. [Abstract / Full text] The central claim of extensive experimental results and significant outperformance is unsupported. The full text following the abstract is a mathematics paper (arXiv:2508.07437 [math.AC]) with no mention of ReLoc, LLMs, code generation, or any experiment. There are no datasets, baseline methods, metrics, ablations, or statistical analyses. Since the paper's value rests on these empirical claims, this is a load-bearing omission.
  2. [Abstract] The revision reward model is load-bearing for the proposed search: the abstract states it 'evaluates code quality based on revision distance to produce fine-grained preferences.' No definition of revision distance, training procedure, data, or validation of this reward model is provided anywhere in the submitted file. Without this, the mechanism guiding the local search cannot be evaluated.
  3. [Abstract] The claimed superiority over 'construction-based tree search' and 'state-of-the-art improvement-based code generation methods' is not accompanied by any specification of the comparison. No baseline names, task suites, evaluation metrics, or significance tests are given. Even if the correct full text were supplied, these comparisons would need to be concrete and reproducible.
minor comments (2)
  1. [Header / Metadata] The arXiv identifier in the supplied full text (2508.07437 [math.AC]) does not match the abstract's paper identifier (2508.07434, cs.CL). The submission appears to contain the wrong PDF; this should be corrected.
  2. [Title] The phrase 'step-by-step' in the title and abstract is not defined. If it is meant to relate to stepwise repair or chain-of-thought style revision, the relationship should be made explicit in the actual paper.

Circularity Check

0 steps flagged

No circularity identifiable: the provided full text is a different paper, so no derivation chain for ReLoc is available to inspect.

full rationale

The task is to detect circularity in the claimed derivation chain of the cs.CL paper 'Let's Revise Step-by-Step' (arXiv:2508.07434). However, the supplied full text is arXiv:2508.07437v1 [math.AC], a commutative algebra paper titled 'Joint Reductions and Mixed Buchsbaum-Rim Multiplicities of Modules and a Joint-Reduction-Number-Zero Theorem' by Katz, Kodiyalam, and Verma. It contains no description of ReLoc, no revision reward model, no code-generation experiments, and no equations corresponding to the abstract's claims. Consequently, there is no concrete equation, fitted parameter, self-citation chain, or defined term that can be exhibited as reducing a prediction to an input. The abstract alone does not define the revision reward model in terms of the evaluation metric, nor does it fit a parameter and call the fitted value a prediction. The central claim of superior performance is unverifiable from the provided materials because the experimental evidence is absent, but absence of evidence is not circularity. Under the hard rule that circularity must be demonstrated by quoting a specific reduction, no such reduction can be found. The document mismatch is a serious verifiability and integrity problem, but it does not warrant a circularity score above 0.

Axiom & Free-Parameter Ledger

1 free parameters · 3 axioms · 0 invented entities

The abstract alone provides no derivations or experimental details. The only identifiable free parameter is the reward model's fitted parameters. The main axioms are unverified assumptions about LLM revision ability and the usefulness of revision distance as a reward signal.

free parameters (1)
  • revision reward model parameters = unknown
    The abstract states that a specialized revision reward model is developed to evaluate code quality based on revision distance. Its internal parameters are presumably fitted to preference data, but no details or values are reported.
axioms (3)
  • domain assumption LLMs can produce useful step-by-step code revisions when prompted
    The method relies on generating neighborhood code revisions with LLMs, which is an assumption about model capability stated in the abstract.
  • domain assumption Local search over revisions converges to better code quality
    The abstract assumes that iteratively updating an incumbent code solution through local revisions leads to improvement, without providing convergence analysis.
  • domain assumption Revision distance is a valid reward signal for code quality
    The proposed reward model scores candidates based on revision distance. This assumes distance from the incumbent correlates with quality improvement, which is not justified in the abstract.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Let's Revise Step-by-Step: A Unified Local Search Framework for Code Generation with LLMs." pith.science (2026). https://pith.science/paper/XX2V66XT

@misc{pith2026250807434,
  author       = {Pith},
  title        = {Pith review of: Let's Revise Step-by-Step: A Unified Local Search Framework for Code Generation with LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XX2V66XT}},
  note         = {Machine review of arXiv:2508.07434}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Large Language Models (LLMs) with inference-time scaling techniques show promise for code generation, yet face notable efficiency and scalability challenges. Construction-based tree-search methods suffer from rapid growth in tree size, high token consumption, and lack of anytime property. In contrast, improvement-based methods offer better performance but often struggle with uninformative reward signals and inefficient search strategies. In this work, we propose \textbf{ReLoc}, a unified local search framework which effectively performs step-by-step code revision. Specifically, ReLoc explores a series of local revisions through four key algorithmic components: initial code drafting, neighborhood code generation, candidate evaluation, and incumbent code updating, each of which can be instantiated with specific decision rules to realize different local search algorithms such as Hill Climbing (HC) or Genetic Algorithm (GA). Furthermore, we develop a specialized revision reward model that evaluates code quality based on revision distance to produce fine-grained preferences that guide the local search toward more promising candidates. Finally, our extensive experimental results demonstrate that our approach achieves superior performance across diverse code generation tasks, significantly outperforming both construction-based tree search as well as the state-of-the-art improvement-based code generation methods.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. AdverMCTS: Combating Pseudo-Correctness in Code Generation via Adversarial Monte Carlo Tree Search

    cs.SE 2026-04 unverdicted novelty 7.0

    AdverMCTS frames code generation as a minimax game where an attacker evolves tests to expose flaws in solver-generated code, yielding more robust outputs than static-test baselines.

  2. Code as Agent Harness

    cs.CL 2026-05 accept novelty 5.0

    A survey that organizes existing work on LLM-based agents around code as the central harness, structured in three layers of interfaces, mechanisms, and multi-agent scaling, with applications across domains and listed ...

Reference graph

Works this paper leans on

57 extracted references · 16 canonical work pages · cited by 2 Pith papers

  1. [1]

    GPT -4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. GPT -4 technical report. arXiv preprint arXiv:2303.08774, 2023

    Pith/arXiv arXiv 2023

  2. [2]

    Openhands: A n open platform for AI software developers as generalist agents

    Xingyao Wang, Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, et al. Openhands: A n open platform for AI software developers as generalist agents. In ICLR, 2024 a

    work page 2024

  3. [3]

    Evaluating large language models trained on code

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021

    Pith/arXiv arXiv 2021

  4. [4]

    Program synthesis with large language models

    Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021

    Pith/arXiv arXiv 2021

  5. [5]

    Less training, more repairing please: R evisiting automated program repair via zero-shot learning

    Chunqiu Steven Xia and Lingming Zhang. Less training, more repairing please: R evisiting automated program repair via zero-shot learning. In ESEC/FSE, pages 959--971, 2022

    work page 2022

  6. [6]

    Impact of code language models on automated program repair

    Nan Jiang, Kevin Liu, Thibaud Lutellier, and Lin Tan. Impact of code language models on automated program repair. In ICSE, pages 1430--1442. IEEE, 2023

    work page 2023

  7. [7]

    Inferfix: E nd-to-end program repair with LLM s

    Matthew Jin, Syed Shahriar, Michele Tufano, Xin Shi, Shuai Lu, Neel Sundaresan, and Alexey Svyatkovskiy. Inferfix: E nd-to-end program repair with LLM s. In ESEC/FSE, pages 1646--1656, 2023

    work page 2023

  8. [8]

    Learning performance-improving code edits

    Alexander Shypula, Aman Madaan, Yimeng Zeng, Uri Alon, Jacob Gardner, Milad Hashemi, Graham Neubig, Parthasarathy Ranganathan, Osbert Bastani, and Amir Yazdanbakhsh. Learning performance-improving code edits. arXiv preprint arXiv:2302.07867, 2023

    Pith/arXiv arXiv 2023

  9. [9]

    Large language models for compiler optimization

    Chris Cummins, Volker Seeker, Dejan Grubisic, Mostafa Elhoushi, Youwei Liang, Baptiste Roziere, Jonas Gehring, Fabian Gloeckle, Kim Hazelwood, Gabriel Synnaeve, et al. Large language models for compiler optimization. arXiv preprint arXiv:2309.07062, 2023

    Pith/arXiv arXiv 2023

  10. [10]

    Tree of thoughts: D eliberate problem solving with large language models

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: D eliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36: 0 11809--11822, 2023

    work page 2023

  11. [11]

    Scaling LLM test-time compute optimally can be more effective than scaling model parameters

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314, 2024

    Pith/arXiv arXiv 2024

  12. [12]

    Alphazero-like tree-search can guide large language model decoding and training

    Xidong Feng, Ziyu Wan, Muning Wen, Stephen Marcus McAleer, Ying Wen, Weinan Zhang, and Jun Wang. Alphazero-like tree-search can guide large language model decoding and training. arXiv preprint arXiv:2309.17179, 2023

    Pith/arXiv arXiv 2023

  13. [13]

    Q*: I mproving multi-step reasoning for LLM s with deliberative planning

    Chaojie Wang, Yanchen Deng, Zhiyi Lyu, Liang Zeng, Jujie He, Shuicheng Yan, and Bo An. Q*: I mproving multi-step reasoning for LLM s with deliberative planning. arXiv preprint arXiv:2406.14283, 2024 b

    Pith/arXiv arXiv 2024

  14. [14]

    Math-shepherd: V erify and reinforce LLM s step-by-step without human annotations

    Peiyi Wang, Lei Li, Zhihong Shao, RX Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: V erify and reinforce LLM s step-by-step without human annotations. arXiv preprint arXiv:2312.08935, 2023

    Pith/arXiv arXiv 2023

  15. [15]

    Let's verify step by step

    Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In ICLR, 2023

    work page 2023

  16. [16]

    Using anytime algorithms in intelligent systems

    Shlomo Zilberstein. Using anytime algorithms in intelligent systems. AI Magazine, 17 0 (3): 0 73--73, 1996

    work page 1996

  17. [17]

    Codetree: A gent-guided tree search for code generation with large language models

    Jierui Li, Hung Le, Yingbo Zhou, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Codetree: A gent-guided tree search for code generation with large language models. arXiv preprint arXiv:2411.04329, 2024 a

    Pith/arXiv arXiv 2024

  18. [18]

    Scattered forest search: S marter code space exploration with LLM s

    Jonathan Light, Yue Wu, Yiyou Sun, Wenchao Yu, Xujiang Zhao, Ziniu Hu, Haifeng Chen, Wei Cheng, et al. Scattered forest search: S marter code space exploration with LLM s. arXiv preprint arXiv:2411.05010, 2024

    Pith/arXiv arXiv 2024

  19. [19]

    What makes large language models reason in (multi-turn) code generation? arXiv preprint arXiv:2410.08105, 2024

    Kunhao Zheng, Juliette Decugis, Jonas Gehring, Taco Cohen, Benjamin Negrevergne, and Gabriel Synnaeve. What makes large language models reason in (multi-turn) code generation? arXiv preprint arXiv:2410.08105, 2024

    Pith/arXiv arXiv 2024

  20. [20]

    Agentless: D emystifying LLM -based software engineering agents

    Chunqiu Steven Xia, Yinlin Deng, Soren Dunn, and Lingming Zhang. Agentless: D emystifying LLM -based software engineering agents. arXiv preprint arXiv:2407.01489, 2024

    Pith/arXiv arXiv 2024

  21. [21]

    Is self-repair a silver bullet for code generation? arXiv preprint arXiv:2306.09896, 2023

    Theo X Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar-Lezama. Is self-repair a silver bullet for code generation? arXiv preprint arXiv:2306.09896, 2023

    Pith/arXiv arXiv 2023

  22. [22]

    Self-alignment for factuality: M itigating hallucinations in LLM s via self-evaluation

    Xiaoying Zhang, Baolin Peng, Ye Tian, Jingyan Zhou, Lifeng Jin, Linfeng Song, Haitao Mi, and Helen Meng. Self-alignment for factuality: M itigating hallucinations in LLM s via self-evaluation. arXiv preprint arXiv:2402.09267, 2024 a

    Pith/arXiv arXiv 2024

  23. [23]

    RethinkMCTS : R efining erroneous thoughts in monte carlo tree search for code generation

    Qingyao Li, Wei Xia, Kounianhua Du, Xinyi Dai, Ruiming Tang, Yasheng Wang, Yong Yu, and Weinan Zhang. RethinkMCTS : R efining erroneous thoughts in monte carlo tree search for code generation. arXiv preprint arXiv:2409.09584, 2024 b

    arXiv 2024

  24. [24]

    Artificial Intelligence: A Modern Approach

    Stuart J Russell and Peter Norvig. Artificial Intelligence: A Modern Approach. pearson, 2016

    work page 2016

  25. [25]

    An Introduction to Genetic Algorithms

    Melanie Mitchell. An Introduction to Genetic Algorithms. MIT press, 1998

    work page 1998

  26. [26]

    Rank analysis of incomplete block designs: I

    Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39 0 (3/4): 0 324--345, 1952

    1952

  27. [27]

    Livecodebench: H olistic and contamination free evaluation of large language models for code

    Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Armando Solar-Lezama, Koushik Sen, and Ion Stoica. Livecodebench: H olistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024

    Pith/arXiv arXiv 2024

  28. [28]

    TACO : T opics in algorithmic code generation dataset

    Rongao Li, Jie Fu, Bo-Wen Zhang, Tao Huang, Zhihong Sun, Chen Lyu, Guang Liu, Zhi Jin, and Ge Li. TACO : T opics in algorithmic code generation dataset. arXiv preprint arXiv:2312.14852, 2023

    Pith/arXiv arXiv 2023

  29. [29]

    General local search methods

    Marc Pirlot. General local search methods. European Journal of Operational Research, 92 0 (3): 0 493--511, 1996

    work page 1996

  30. [30]

    Policy filtration in rlhf to fine-tune llm for code generation

    Wei Shen and Chuheng Zhang. Policy filtration in rlhf to fine-tune llm for code generation. arXiv preprint arXiv:2409.06957, 2024

    Pith/arXiv arXiv 2024

  31. [31]

    Training language models to follow instructions with human feedback

    Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. In NeurIPS, pages 27730--27744, 2022

    work page 2022

  32. [32]

    Iterated local search: F ramework and applications

    Helena Ramalhinho Louren c o, Olivier C Martin, and Thomas St \"u tzle. Iterated local search: F ramework and applications. In Handbook of Metaheuristics, pages 129--168. Springer, 2018

    work page 2018

  33. [33]

    Course of Theoretical Physics

    Lev Davidovich Landau and Evgenii Mikhailovich Lifshitz. Course of Theoretical Physics. Elsevier, 2013

    work page 2013

  34. [34]

    Simulated annealing: F rom basics to applications

    Daniel Delahaye, Supatcha Chaimatanan, and Marcel Mongeau. Simulated annealing: F rom basics to applications. In Handbook of Metaheuristics, pages 1--35. Springer, 2018

    work page 2018

  35. [35]

    Measuring coding challenge competence with APPS

    Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, Samir Puranik, Horace He, Dawn Song, et al. Measuring coding challenge competence with APPS . arXiv preprint arXiv:2105.09938, 2021

    Pith/arXiv arXiv 2021

  36. [36]

    TRL : Transformer reinforcement learning

    Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. TRL : Transformer reinforcement learning. https://github.com/huggingface/trl, 2020

    work page 2020

  37. [37]

    Reasoning with language model is planning with world model

    Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305.14992, 2023

    Pith/arXiv arXiv 2023

  38. [38]

    Reflexion: L anguage agents with verbal reinforcement learning

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: L anguage agents with verbal reinforcement learning. NeurIPS, 36: 0 8634--8652, 2023

    work page 2023

  39. [39]

    Planning in natural language improves LLM search for code generation

    Evan Wang, Federico Cassano, Catherine Wu, Yunfeng Bai, Will Song, Vaskar Nath, Ziwen Han, Sean Hendryx, Summer Yue, and Hugh Zhang. Planning in natural language improves LLM search for code generation. arXiv preprint arXiv:2409.03733, 2024 c

    Pith/arXiv arXiv 2024

  40. [40]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

    Pith/arXiv arXiv 2021

  41. [41]

    Outcome-refining process supervision for code generation

    Zhuohao Yu, Weizheng Gu, Yidong Wang, Zhengran Zeng, Jindong Wang, Wei Ye, and Shikun Zhang. Outcome-refining process supervision for code generation. arXiv preprint arXiv:2412.15118, 2024

    Pith/arXiv arXiv 2024

  42. [42]

    Skywork-reward: B ag of tricks for reward modeling in LLM s

    Chris Yuhao Liu, Liang Zeng, Jiacai Liu, Rui Yan, Jujie He, Chaojie Wang, Shuicheng Yan, Yang Liu, and Yahui Zhou. Skywork-reward: B ag of tricks for reward modeling in LLM s. arXiv preprint arXiv:2410.18451, 2024

    Pith/arXiv arXiv 2024

  43. [43]

    Codexglue: A machine learning benchmark dataset for code understanding and generation

    Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin Clement, Dawn Drain, Daxin Jiang, Duyu Tang, et al. Codexglue: A machine learning benchmark dataset for code understanding and generation. arXiv preprint arXiv:2102.04664, 2021

    Pith/arXiv arXiv 2021

  44. [44]

    Pangu-coder: P rogram synthesis with function-level language modeling

    Fenia Christopoulou, Gerasimos Lampouras, Milan Gritta, Guchun Zhang, Yinpeng Guo, Zhongqi Li, Qi Zhang, Meng Xiao, Bo Shen, Lin Li, et al. Pangu-coder: P rogram synthesis with function-level language modeling. arXiv preprint arXiv:2207.11280, 2022

    Pith/arXiv arXiv 2022

  45. [45]

    DeepSeek-Coder : W hen the large language model meets programming--the rise of code intelligence

    Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. DeepSeek-Coder : W hen the large language model meets programming--the rise of code intelligence. arXiv preprint arXiv:2401.14196, 2024

    Pith/arXiv arXiv 2024

  46. [46]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.12186, 2024

    Pith/arXiv arXiv 2024

  47. [47]

    Language agent tree search unifies reasoning acting and planning in language models

    Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman, Haohan Wang, and Yu-Xiong Wang. Language agent tree search unifies reasoning acting and planning in language models. arXiv preprint arXiv:2310.04406, 2023

    Pith/arXiv arXiv 2023

  48. [48]

    Debug like a human: A large language model debugger via verifying runtime execution step-by-step

    Li Zhong, Zilong Wang, and Jingbo Shang. Debug like a human: A large language model debugger via verifying runtime execution step-by-step. arXiv preprint arXiv:2402.16906, 2024

    Pith/arXiv arXiv 2024

  49. [49]

    Codeagent: E nhancing code generation with tool-integrated agent systems for real-world repo-level coding challenges

    Kechi Zhang, Jia Li, Ge Li, Xianjie Shi, and Zhi Jin. Codeagent: E nhancing code generation with tool-integrated agent systems for real-world repo-level coding challenges. arXiv preprint arXiv:2401.07339, 2024 b

    Pith/arXiv arXiv 2024

  50. [50]

    Swe-bench-java: A github issue resolving benchmark for java

    Daoguang Zan, Zhirong Huang, Ailun Yu, Shaoxin Lin, Yifan Shi, Wei Liu, Dong Chen, Zongshuai Qi, Hao Yu, Lei Yu, et al. Swe-bench-java: A github issue resolving benchmark for java. arXiv preprint arXiv:2408.14354, 2024

    Pith/arXiv arXiv 2024

  51. [51]

    Rlaif vs

    Harrison Lee, Samrat Phatale, Hassan Mansoor, Thomas Mesnard, Johan Ferret, Kellie Lu, Colton Bishop, Ethan Hall, Victor Carbune, Abhinav Rastogi, et al. Rlaif vs. rlhf: Scaling reinforcement learning from human feedback with ai feedback. arXiv preprint arXiv:2309.00267, 2023

    Pith/arXiv arXiv 2023

  52. [52]

    The lessons of developing process reward models in mathematical reasoning

    Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. The lessons of developing process reward models in mathematical reasoning. arXiv preprint arXiv:2501.07301, 2025

    Pith/arXiv arXiv 2025

  53. [53]

    Technical report: Enhancing llm reasoning with reward-guided tree search

    Jinhao Jiang, Zhipeng Chen, Yingqian Min, Jie Chen, Xiaoxue Cheng, Jiapeng Wang, Yiru Tang, Haoxiang Sun, Jia Deng, Wayne Xin Zhao, et al. Technical report: Enhancing llm reasoning with reward-guided tree search. arXiv preprint arXiv:2411.11694, 2024

    Pith/arXiv arXiv 2024

  54. [54]

    Generative reward models

    Dakota Mahan, Duy Van Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fr \"a nken, Chelsea Finn, and Alon Albalak. Generative reward models. arXiv preprint arXiv:2410.12832, 2024

    Pith/arXiv arXiv 2024

  55. [55]

    Llm critics help catch llm bugs

    Nat McAleese, Rai Michael Pokorny, Juan Felipe Ceron Uribe, Evgenia Nitishinskaya, Maja Trebacz, and Jan Leike. Llm critics help catch llm bugs. arXiv preprint arXiv:2407.00215, 2024

    Pith/arXiv arXiv 2024

  56. [56]

    Generative verifiers: Reward modeling as next-token prediction

    Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction. arXiv preprint arXiv:2408.15240, 2024 c

    Pith/arXiv arXiv 2024

  57. [57]

    Rm-r1: Reward modeling as reasoning

    Xiusi Chen, Gaotang Li, Ziqi Wang, Bowen Jin, Cheng Qian, Yu Wang, Hongru Wang, Yu Zhang, Denghui Zhang, Tong Zhang, et al. Rm-r1: Reward modeling as reasoning. arXiv preprint arXiv:2505.02387, 2025

    arXiv 2025

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.