REVIEW 15 cited by
REFINER: Reasoning Feedback on Intermediate Representations
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Language models (LMs) have recently shown remarkable performance on reasoning tasks by explicitly generating intermediate inferences, e.g., chain-of-thought prompting. However, these intermediate inference steps may be inappropriate deductions from the initial context and lead to incorrect final predictions. Here we introduce REFINER, a framework for finetuning LMs to explicitly generate intermediate reasoning steps while interacting with a critic model that provides automated feedback on the reasoning. Specifically, the critic provides structured feedback that the reasoning LM uses to iteratively improve its intermediate arguments. Empirical evaluations of REFINER on three diverse reasoning tasks show significant improvements over baseline LMs of comparable scale. Furthermore, when using GPT-3.5 or ChatGPT as the reasoner, the trained critic significantly improves reasoning without finetuning the reasoner. Finally, our critic model is trained without expensive human-in-the-loop data but can be substituted with humans at inference time.
Forward citations
Cited by 15 Pith papers
-
Grammar-Guided Evolutionary Search for Discrete Prompt Optimisation
A grammar-guided evolutionary search that composes prompt edits outperformed PromptWizard, OPRO, and RL-Prompt on small LLMs across four domain-specific tasks.
-
Self-guided Knowledgeable Network of Thoughts: Amplifying Reasoning with Large Language Models
A large language model can plan and write an executable network of elementary reasoning steps for itself, and this self-guided workflow beats prior prompting schemes on sorting, arithmetic, and language counting tasks.
-
Refining Answer Distributions for Improved Large Language Model Reasoning
RAD iteratively refines a distribution over answers by marginalizing over refinement samples, improving accuracy on six arithmetic benchmarks over self-consistency and hint-based prompting.
-
Enhancing Relation Extraction via Supervised Rationale Verification and Feedback
A supervised rationale verifier trained on contrastively collected biased and unbiased rationales improves few-shot relation extraction by retrieving corrective demonstrations for the LLM.
-
A MARL Centered Reference Architecture for Large Language Model Augmentation in Smart Manufacturing
The paper proposes a MARL-centered three-layer reference architecture for LLM augmentation in smart manufacturing, with a conditional allocation: MARL for frequent coordination, LLMs for semantic, reward, and planning roles.
-
CS-Agent: LLM-based Community Search via Dual-agent Collaboration
CS-Agent, a Solver-Validator two-agent dialogue with a Decider selector, improves LLM community search on synthetic graphs, and GraphCS is a new benchmark for measuring it.
-
R4ec: A Reasoning, Reflection, and Refinement Framework for Recommendation Systems
R4ec trains a small reflection model to critique and refine LLM-generated user and item knowledge, which then improves downstream recommendation accuracy.
-
Boosting LLM Reasoning via Spontaneous Self-Correction
SPOC trains LLMs to interleave self-verification and solution attempts in a single pass, reporting gains on math benchmarks, though most gains come from stronger first attempts.
-
Understanding the Dark Side of LLMs' Intrinsic Self-Correction
Intrinsic self-correction makes state-of-the-art LLMs overturn correct answers across four task types, and simple question repetition or tiny fine-tuning reduces this damage.
-
Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning
A DPO-trained VLM critic that critiques and iteratively refines a reasoning VLM improves accuracy on several multimodal benchmarks, with large gains on MathVista and RealWorldQA.
-
Can Compressed LLMs Truly Act? An Empirical Evaluation of Agentic Capabilities in LLM Compression
ACBench tests compressed LLMs on agentic tasks and finds 4-bit quantization keeps tool use and workflow generation strong while hurting real-world application performance.
-
Are Retrials All You Need? Enhancing Large Language Model Reasoning Without Verbalized Feedback
Retrying failed attempts with fresh samples, called retrials without feedback, is a cheap and effective way to boost LLM reasoning, often outperforming complex methods like tree-of-thoughts.
-
Recursive Decomposition of Logical Thoughts: Framework for Superior Reasoning and Knowledge Propagation in Large Language Models
A prompting framework that recursively decomposes reasoning tasks and self-scores candidate thoughts is reported to improve LLM accuracy on math and letter-concatenation benchmarks, though the headline improvement is ...
-
Towards Intrinsic Self-Correction Enhancement in Monte Carlo Tree Search Boosted Reasoning via Iterative Preference Learning
Adding a self-correction RL stage before MCTS-DPO improves arithmetic reasoning accuracy by 2.00 to 4.94 points on GSM8K and MATH.
-
From Critique to Clarity: A Pathway to Faithful and Personalized Code Explanations with Large Language Models
An iterative two-loop LLM pipeline (a faithfulness loop with execution-based checks and a personalization loop with a role-playing judge) produces code explanations that score higher on automatic metrics than simpler ...
Discussion (0). Continue with ORCID to comment.