REVIEW 6 cited by
AutoPSV: Automated Process-Supervised Verifier
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
In this work, we propose a novel method named \textbf{Auto}mated \textbf{P}rocess-\textbf{S}upervised \textbf{V}erifier (\textbf{\textsc{AutoPSV}}) to enhance the reasoning capabilities of large language models (LLMs) by automatically annotating the reasoning steps. \textsc{AutoPSV} begins by training a verification model on the correctness of final answers, enabling it to generate automatic process annotations. This verification model assigns a confidence score to each reasoning step, indicating the probability of arriving at the correct final answer from that point onward. We detect relative changes in the verification's confidence scores across reasoning steps to automatically annotate the reasoning process, enabling error detection even in scenarios where ground truth answers are unavailable. This alleviates the need for numerous manual annotations or the high computational costs associated with model-induced annotation approaches. We experimentally validate that the step-level confidence changes learned by the verification model trained on the final answer correctness can effectively identify errors in the reasoning steps. We demonstrate that the verification model, when trained on process annotations generated by \textsc{AutoPSV}, exhibits improved performance in selecting correct answers from multiple LLM-generated outputs. Notably, we achieve substantial improvements across five datasets in mathematics and commonsense reasoning. The source code of \textsc{AutoPSV} is available at \url{https://github.com/rookie-joe/AutoPSV}.
Forward citations
Cited by 6 Pith papers
-
SCOPE: Compress Mathematical Reasoning Steps for Efficient Automated Process Annotation
SCOPE compresses sampled math reasoning traces into a prefix tree of AST-normalized code steps, yielding PRM training labels at O(N) cost and improving Best-of-N and ProcessBench scores over automated baselines.
-
A Simple "Try Again" Can Elicit Multi-Turn LLM Reasoning
Multi-turn RL with only unary 'try again' feedback improves multi-turn reasoning accuracy by up to 14% while preserving single-turn performance.
-
Ensembling Large Language Models with Process Reward-Guided Tree Search for Better Complex Reasoning
LE-MCTS searches over reasoning steps generated by different LLMs, using a process reward model to select the best trajectory, and reports accuracy gains on math benchmarks such as MATH and MathQA.
-
UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models
UAlign improves LLM factuality alignment by adding predicted confidence and semantic entropy as input features to prompts and the reward model, helping the model answer known questions and refuse unknown ones.
-
Free Process Rewards without Process Labels
A process reward model can be obtained from an outcome reward model by parameterizing rewards as beta log(pi_theta/pi_ref) and taking per-token log-ratio differences as step scores.
-
AceMath: Advancing Frontier Math Reasoning with Post-Training and Reward Modeling
AceMath models use general-then-math fine-tuning and a listwise-trained reward model to report state-of-the-art math reasoning scores.
Discussion (0). Continue with ORCID to comment.