Pith. sign in

REVIEW 23 cited by

Mutual Reasoning Makes Smaller LLMs Stronger Problem-Solvers

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2408.06195 v1 pith:MDR67XGR submitted 2024-08-12 cs.CL

Zhenting Qi , Mingyuan Ma , Jiahang Xu , Li Lyna Zhang , Fan Yang , Mao Yang This is my paper · ORCID
classification cs.CL
keywords reasoningrstarmutualtargetcapabilitiesgsm8kmodelsself-play
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

This paper introduces rStar, a self-play mutual reasoning approach that significantly improves reasoning capabilities of small language models (SLMs) without fine-tuning or superior models. rStar decouples reasoning into a self-play mutual generation-discrimination process. First, a target SLM augments the Monte Carlo Tree Search (MCTS) with a rich set of human-like reasoning actions to construct higher quality reasoning trajectories. Next, another SLM, with capabilities similar to the target SLM, acts as a discriminator to verify each trajectory generated by the target SLM. The mutually agreed reasoning trajectories are considered mutual consistent, thus are more likely to be correct. Extensive experiments across five SLMs demonstrate rStar can effectively solve diverse reasoning problems, including GSM8K, GSM-Hard, MATH, SVAMP, and StrategyQA. Remarkably, rStar boosts GSM8K accuracy from 12.51% to 63.91% for LLaMA2-7B, from 36.46% to 81.88% for Mistral-7B, from 74.53% to 91.13% for LLaMA3-8B-Instruct. Code will be available at https://github.com/zhentingqi/rStar.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 23 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. TimeHC-RL: Temporal-aware Hierarchical Cognitive Reinforcement Learning for Enhancing LLMs' Social Intelligence

    cs.CL 2025-05 conditional novelty 7.0 of 10

    A temporal-aware hierarchical reinforcement learning method improves a 7B LLM's social reasoning enough to rival DeepSeek-R1 and OpenAI-O3 on in-domain theory-of-mind benchmarks.

  2. VLM-R$^3$: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought

    cs.CV 2025-05 conditional novelty 7.0 of 10

    VLM-R3 uses reinforcement learning to teach a multimodal model when and where to crop an image during its chain-of-thought, improving scores on several visual reasoning benchmarks.

  3. Are They All Good? Evaluating the Quality of CoTs in LLM-based Code Generation

    cs.SE 2025-07 reject novelty 6.0 of 10

    An empirical study of 1,023 CoT-code pairs shows that 76.4% of LLM-generated CoTs are low quality and that CoT correctness does not guarantee code correctness.

  4. MiCoTA: Bridging the Learnability Gap with Intermediate CoT and Teacher Assistants

    cs.CL 2025-07 conditional novelty 6.0 of 10

    Training small language models on intermediate-length reasoning chains from a merged mid-sized teacher assistant improves their math reasoning scores over direct distillation from a large teacher.

  5. Chain of Methodologies: Scaling Test Time Computation without Training

    cs.CL 2025-06 reject novelty 6.0 of 10

    A prompt framework that interleaves methodology selection with reasoning steps improves LLM math and QA accuracy when combined with a Python interpreter, but the benefit of the methodology selection itself is small an...

  6. Token Signature: Predicting Chain-of-Thought Gains with Token Decoding Feature in Large Language Models

    cs.CL 2025-06 conditional novelty 6.0 of 10

    The monotonicity of token probabilities during initial decoding predicts chain-of-thought gains, enabling dynamic selection between CoT and direct answers.

  7. LLM-First Search: Self-Guided Exploration of the Solution Space

    cs.AI 2025-06 conditional novelty 6.0 of 10

    LLM-First Search, where the model itself decides whether to continue or backtrack during reasoning, outperforms MCTS, BestFS, and ToT-BFS on harder Countdown and Sudoku tasks while using fewer tokens.

  8. ProtInvTree: Deliberate Protein Inverse Folding with Reward-guided Tree Search

    q-bio.BM 2025-06 conditional novelty 6.0 of 10

    A reward-guided tree search over a frozen protein language model designs diverse sequences that score higher on ESMFold-based self-consistency benchmarks than existing inverse folding methods.

  9. MMATH: A Multilingual Benchmark for Mathematical Reasoning

    cs.CL 2025-05 conditional novelty 6.0 of 10

    A new multilingual math benchmark shows that reasoning models often respond in the wrong language, and English-reasoning training improves both accuracy and language consistency.

  10. Self-Critique Guided Iterative Reasoning for Multi-hop Question Answering

    cs.CL 2025-05 conditional novelty 6.0 of 10

    SiGIR trains a language model to decompose multi-hop questions, self-critique each retrieval and reasoning step, and use cumulative self-rewards in a beam search, beating prior methods by up to 14.4 F1 points on MuSiQue.

  11. Satori: Reinforcement Learning with Chain-of-Action-Thought Enhances LLM Reasoning via Autoregressive Search

    cs.CL 2025-02 conditional novelty 6.0 of 10

    Training a 7B LLM on 10K formatted demonstrations plus large-scale RL with a restart-and-explore strategy produces a single model that searches over reasoning steps and improves math and out-of-domain benchmarks.

  12. LongDPO: Unlock Better Long-form Generation Abilities for LLMs via Critique-augmented Stepwise Information

    cs.CL 2025-02 conditional novelty 6.0 of 10

    LongDPO uses MCTS-collected, critique-refined stepwise preference pairs with step-level DPO to improve long-form generation quality and length adherence.

  13. Reward-Guided Speculative Decoding for Efficient LLM Reasoning

    cs.CL 2025-01 conditional novelty 6.0 of 10

    A reward-guided threshold decides whether to accept a draft model's step or call a larger target model, cutting inference cost up to 4.4x while slightly improving accuracy on reasoning benchmarks.

  14. Scaling Inference-Efficient Language Models

    cs.LG 2025-01 conditional novelty 6.0 of 10

    An inference-aware scaling law that includes model aspect ratio ranks model shapes by loss and latency, producing a 1B model that is 1.8x faster without losing accuracy.

  15. MILES: Modular Instruction Memory with Learnable Selection for Self-Improving LLM Reasoning

    cs.CL 2026-07 unverdicted novelty 5.0 of 10

    MILES dynamically expands step-wise memory with learnable selection heads that rerank candidates and guide reasoning, improving LLM test-time performance under limited supervision.

  16. SVSR: A Self-Verification and Self-Rectification Paradigm for Multimodal Reasoning

    cs.AI 2026-04 unverdicted novelty 5.0 of 10

    SVSR trains multimodal models to verify and correct their own reasoning using a preference dataset, supervised fine-tuning, and semi-online DPO with a teacher model.

  17. Thinking Before You Speak: A Proactive Test-time Scaling Approach

    cs.CL 2025-08 reject novelty 5.0 of 10

    TBYS inserts generated situation-and-goal insights between reasoning steps and reports math accuracy gains, but the insight library is filtered using test-set answers.

  18. Boosting LLM's Molecular Structure Elucidation with Knowledge Enhanced Tree Search Reasoning

    cs.CL 2025-06 conditional novelty 5.0 of 10

    K-MSE adds a substructure knowledge base, a learned molecule-spectrum scorer, and Monte Carlo tree search, lifting LLM exact-match accuracy on MolPuzzle from 3.7% to 27.3% (GPT-4o-mini) and from 27.8% to 57.8% (GPT-4o).

  19. Fast Quiet-STaR: Thinking Without Thought Tokens

    cs.CL 2025-05 conditional novelty 5.0 of 10

    Fast Quiet-STaR uses a curriculum to compress Quiet-STaR's token-level thoughts and an RL stage to remove them entirely, improving accuracy on four benchmarks at the same or lower inference cost.

  20. On the Emergence of Thinking in LLMs I: Searching for the Right Intuition

    cs.AI 2025-02 conditional novelty 5.0 of 10

    A post-training framework called RLSP, decoupling exploration from correctness rewards, induces backtracking and self-correction in LLMs and improves math reasoning performance.

  21. Policy Guided Tree Search for Enhanced LLM Reasoning

    cs.LG 2025-02 conditional novelty 5.0 of 10

    PGTS trains a graph-transformer policy via PPO to guide tree search over LLM reasoning steps, reporting improved accuracy and lower token use than chain-of-thought and MCTS baselines.

  22. Scalable, Symbiotic, AI and Non-AI Agent Based Parallel Discrete Event Simulations

    cs.CL 2025-05 conditional novelty 4.0 of 10

    PDES orchestration of small language models with non-AI verifier agents raises accuracy on four toy tasks from about 23 percent to 68 percent, with the verifiers supplying most of the correctness.

  23. The Hitchhiker's Guide to Agentic AI: From Foundations to Systems

    cs.AI 2026-06 unverdicted novelty 2.0 of 10

    A survey-style reference book mapping the full agentic-AI stack from transformer internals to production deployment, with no new research result.

Pith tools