REVIEW 9 cited by
Enhancing LLM Reasoning with Reward-guided Tree Search
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Recently, test-time scaling has garnered significant attention from the research community, largely due to the substantial advancements of the o1 model released by OpenAI. By allocating more computational resources during the inference phase, large language models~(LLMs) can extensively explore the solution space by generating more thought tokens or diverse solutions, thereby producing more accurate responses. However, developing an o1-like reasoning approach is challenging, and researchers have been making various attempts to advance this open area of research. In this paper, we present a preliminary exploration into enhancing the reasoning abilities of LLMs through reward-guided tree search algorithms. This framework is implemented by integrating the policy model, reward model, and search algorithm. It is primarily constructed around a tree search algorithm, where the policy model navigates a dynamically expanding tree guided by a specially trained reward model. The implemented framework is denoted as \textbf{STILL-1}. We thoroughly explore various design considerations necessary for implementing this framework and provide a detailed report of the technical aspects. To assess the effectiveness of our approach, we focus on mathematical reasoning tasks and conduct extensive evaluations on four challenging datasets, significantly enhancing the reasoning abilities of LLMs.
Forward citations
Cited by 9 Pith papers
-
Sticker-TTS: Learn to Utilize Historical Experience with a Sticker-driven Test-Time Scaling Framework
Sticker-TTS uses three collaborating models to distill and reuse compact summaries (stickers) of past reasoning attempts, improving math benchmark accuracy at a claimed equivalent inference cost.
-
Com$^2$: A Causal-Guided Benchmark for Exploring Complex Commonsense Reasoning in Large Language Models
Com2 is a causal-graph-guided benchmark with 3,754 questions showing that LLMs struggle with complex commonsense reasoning, particularly on intervention and transition tasks.
-
TreeThink: A Modular Tree Search Library for Mathematical Reasoning with LLMs
TreeThink provides a modular, asynchronous tree-search library for neural theorem proving with unified REPL clients for Lean, Rocq, and Isabelle and up to 6.3× wall-clock speedup.
-
Let's Revise Step-by-Step: A Unified Local Search Framework for Code Generation with LLMs
The abstract claims a new local search framework for code generation, but the manuscript body is a different mathematics paper.
-
Ctrl-Z Sampling: Scaling Diffusion Sampling with Controlled Random Zigzag Explorations
Ctrl-Z Sampling improves text-to-image outputs by adaptively rolling back and re-exploring when a reward model flags a quality plateau, at roughly 3 to 9 times the usual compute.
-
VReST: Enhancing Reasoning in Large Vision-Language Models through Tree Search and Self-Reward Mechanism
VReST combines Monte Carlo tree search with a self-reward signal inside a vision-language model to get higher accuracy than CoT, ToT, or voting baselines on MathVista, MathVision, and CharXiv, while spending several t...
-
Why Does Reasoning Length Converge? Unveiling the Underfitting-Overfitting Trade-off in Chain-of-Thought
LLM chain-of-thought length converges to an optimum because of an underfitting-overfitting tradeoff, formalized in a continuous reasoning-space framework and tested with RL.
-
From Trial-and-Error to Improvement: A Systematic Analysis of LLM Exploration Mechanisms in RLVR
A systematic analysis of LLM exploration in RLVR, introducing capability-boundary metrics and examining entropy-performance exchange across training stages and token levels.
-
Evaluation is All You Need: Strategic Overclaiming of LLM Reasoning Capabilities Through Evaluation Design
Evaluation conditions like seed, dataset version, and answer ordering cause multi-point benchmark score swings in DeepSeek-R1-Distill and related reasoning models, undermining reliable comparison.
Discussion (0). Sign in to comment.