REVIEW 5 cited by
Self-Discover: Large Language Models Self-Compose Reasoning Structures
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
We introduce SELF-DISCOVER, a general framework for LLMs to self-discover the task-intrinsic reasoning structures to tackle complex reasoning problems that are challenging for typical prompting methods. Core to the framework is a self-discovery process where LLMs select multiple atomic reasoning modules such as critical thinking and step-by-step thinking, and compose them into an explicit reasoning structure for LLMs to follow during decoding. SELF-DISCOVER substantially improves GPT-4 and PaLM 2's performance on challenging reasoning benchmarks such as BigBench-Hard, grounded agent reasoning, and MATH, by as much as 32% compared to Chain of Thought (CoT). Furthermore, SELF-DISCOVER outperforms inference-intensive methods such as CoT-Self-Consistency by more than 20%, while requiring 10-40x fewer inference compute. Finally, we show that the self-discovered reasoning structures are universally applicable across model families: from PaLM 2-L to GPT-4, and from GPT-4 to Llama2, and share commonalities with human reasoning patterns.
Forward citations
Cited by 5 Pith papers
-
Node-as-Agent: Graph Agentic Network
A node-as-agent framework where a frozen LLM plans each node's local and global message passing achieves competitive Cora accuracy without training, but uses per-dataset prompt selection and leaves label-leakage quest...
-
Direct Behavior Optimization: Unlocking the Potential of Lightweight LLMs
A gradient-free Monte Carlo tree search over JSON key-step plans produces few-shot demonstrations that let LLaMA3-8B and LLaMA3.2-3B outperform GPT-3.5 on most of seven BIG-Bench Hard tasks.
-
Feature Engineering for Agents: An Adaptive Cognitive Architecture for Interpretable ML Monitoring
CAMA applies a three-step feature engineering procedure to LLM agents and reports 55 to 92 percent accuracy on ML monitoring report questions, outperforming six baselines.
-
Self-Reasoning Language Models: Unfold Hidden Reasoning Chains with Few Reasoning Catalyst
Self-Reasoning Language Models use 1,000 seed catalyst rationales to teach a model to expand its own reasoning chains and iteratively self-improve, gaining about 2.5 points on average across five benchmarks.
-
Mixture of Reasonings: Teach Large Language Models to Reason with Adaptive Strategies
MoR fine-tunes Qwen2.5 on GPT-4o-selected reasoning templates, claiming up to 13.5% accuracy gains, but the reported gains are not robustly supported.
Discussion (0). Continue with ORCID to comment.