REVIEW 14 cited by
Language Models are Hidden Reasoners: Unlocking Latent Reasoning Capabilities via Self-Rewarding
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Large language models (LLMs) have shown impressive capabilities, but still struggle with complex reasoning tasks requiring multiple steps. While prompt-based methods like Chain-of-Thought (CoT) can improve LLM reasoning at inference time, optimizing reasoning capabilities during training remains challenging. We introduce LaTent Reasoning Optimization (LaTRO), a principled framework that formulates reasoning as sampling from a latent distribution and optimizes it via variational approaches. LaTRO enables LLMs to concurrently improve both their reasoning process and ability to evaluate reasoning quality, without requiring external feedback or reward models. We validate LaTRO through experiments on GSM8K and ARC-Challenge datasets using multiple model architectures. On GSM8K, LaTRO improves zero-shot accuracy by an average of 12.5% over base models and 9.6% over supervised fine-tuning across Phi-3.5-mini, Mistral-7B, and Llama-3.1-8B. Our findings suggest that pre-trained LLMs possess latent reasoning capabilities that can be unlocked and enhanced through our proposed optimization approach in a self-improvement manner. The code of LaTRO is available at \url{https://github.com/SalesforceAIResearch/LaTRO}.
Forward citations
Cited by 14 Pith papers
-
Recursive Self-Improvement in AI: From Bounded Self-Refinement to Autonomous Research Loops
A survey of 1,250 papers organizes AI self-improvement along two axes—what is improved and loop closure—finding that demonstrated self-improvement strength tracks a verification hierarchy from formal verifiers down to...
-
LatentChem: From Textual CoT to Latent Thinking in Chemical Reasoning
LatentChem reasons in continuous latent space for chemistry, achieving a 59.88% non-tie win rate over explicit CoT on ChemCoTBench with a 10.84x average reduction in reasoning overhead.
-
AdaThink-Med: Optimizing Inference-Time Compute for Medical Reasoning via Uncertainty Quantification
An entropy-guided reward that shortens answers to easy, confident questions and extends reasoning on hard ones reduces medical QA tokens by 4.7x to 6.4x with minimal accuracy loss.
-
CogDual: Enhancing Dual Cognition of LLMs via Reinforcement Learning with Implicit Rule-Based Rewards
A role-playing LLM that reasons about the scene and its own state before responding, trained with two semantic rewards, beats stronger baselines on role-play benchmarks.
-
Reinforcing General Reasoning without Verifiers
VeriFree trains LLMs with RL by maximizing the likelihood of the reference answer after generated reasoning, matching verifier-based RL without any verifier.
-
LARES: Latent Reasoning for Sequential Recommendation
LARES applies depth-recurrent latent reasoning to sequential recommendation, refining all item tokens at each step, and reports consistent gains across four Amazon benchmarks.
-
MetaSC: Test-Time Safety Specification Optimization for Language Models
MetaSC improves language model safety by using a meta-critic to iteratively rewrite the safety specification that guides self-critique at inference time.
-
BRiTE: Bootstrapping Reinforced Thinking Process to Enhance Language Model Reasoning
BRiTE is an EM-style algorithm that uses RL to sample high-likelihood reasoning chains and fine-tunes the LLM on them, with a theoretical 1/T convergence rate and mixed empirical gains.
-
The Latent Space: Foundation, Evolution, Mechanism, Ability, and Outlook
A large survey organizes latent-space work in language-based models by foundation, evolution, four mechanisms, seven abilities, and open challenges.
-
Coupled Variational Reinforcement Learning for Language Model General Reasoning
CoVRL trains an LLM on a mixture of question-only and answer-guided reasoning traces, using the model's own answer probability as reward, and reports consistent gains on math and general-reasoning benchmarks.
-
Implicit Reasoning in Large Language Models: A Comprehensive Survey
A survey organizing implicit (silent) reasoning in LLMs into three execution paradigms, plus evidence, benchmarks, and challenges.
-
Fast on the Easy, Deep on the Hard: Efficient Reasoning via Powered Length Penalty
Reward shaping with a powered length penalty makes LLMs answer easy questions with far fewer tokens while preserving or slightly improving accuracy on hard math benchmarks.
-
Iterative Self-Incentivization Empowers Large Language Models as Agentic Searchers
EXSEARCH trains LLMs for agentic search by treating search trajectories as latent variables and optimizing a weighted likelihood via expectation-maximization, yielding gains on NQ, HotpotQA, MuSiQue, and 2WikiQA.
-
Position Paper: Bounded Alignment: What (Not) To Expect From AGI Agents
The paper argues that perfect alignment of general AI is impossible in principle and proposes 'bounded alignment' as the realistic safety goal.
Discussion (0). Continue with ORCID to comment.