REVIEW 7 cited by
Achieving >97% on GSM8K: Deeply Understanding the Problems Makes LLMs Better Solvers for Math Word Problems
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Chain-of-Thought (CoT) prompting has enhanced the performance of Large Language Models (LLMs) across various reasoning tasks. However, CoT still falls short in dealing with complex math word problems, as it usually suffers from three pitfalls: semantic misunderstanding errors, calculation errors, and step-missing errors. Prior studies involve addressing the calculation errors and step-missing errors, but neglect the semantic misunderstanding errors, which is the major factor limiting the reasoning performance of LLMs. To this end, we propose a simple-yet-effective method, namely Deeply Understanding the Problems (DUP), to improve the LLMs' math problem-solving ability by addressing semantic misunderstanding errors. The core of our method is to encourage the LLMs to deeply understand the problems and extract the key problem-solving information used for better reasoning. Extensive experiments on 10 diverse reasoning benchmarks show that our DUP method consistently outperforms the other counterparts by a large margin. More encouragingly, DUP achieves a new SOTA result on the GSM8K benchmark, with an accuracy of 97.1% under the zero-shot setting.
Forward citations
Cited by 7 Pith papers
-
Resolving Knowledge Conflicts in Domain-specific Data Selection: A Case Study on Medical Instruction-tuning
KDS selects instruction-tuning data by measuring knowledge alignment (NLI entailment vs reference) and knowledge consistency (cluster entropy of sampled responses), and reports gains on medical QA benchmarks.
-
Artificial Intelligence for Mathematical Reasoning: An Integrated Survey of Language Models, Neuro-symbolic Systems, and Verified Discovery
An integrated survey organizing AI mathematical reasoning into informal, formal, discovery, and technique axes while cataloging benchmarks and assessing failure modes.
-
Robust Knowledge Editing via Explicit Reasoning Chains for Distractor-Resilient Multi-Hop QA
Teaching an LLM to emit a fixed four-stage reasoning chain during fine-tuning makes single-pass multi-hop knowledge editing robust to distractor facts.
-
Soft Guidance Starts to Outperform CoT Prompting as LLMs Improve
On GSM8K, zero-shot free-form generation beats few-shot CoT prompting for Mathstral, Qwen2.5, and Llama-3.1, suggesting standard CoT baselines can underestimate modern reasoning models.
-
DuaShepherd: Integrating Stepwise Correctness and Potential Rewards for Mathematical Reasoning
A reward model that multiplies stepwise correctness and potential scores improves best-of-N verification accuracy for math reasoning.
-
Revisiting Overthinking in Long Chain-of-Thought from the Perspective of Self-Doubt
A 'check the question, then answer concisely' prompt reduces token use and often improves accuracy in reasoning LLMs, but the self-doubt measure is based on an unvalidated LLM judge.
-
A Survey on Large Language Models for Mathematical Reasoning
Recent advances in LLM mathematical reasoning are organized into comprehension and generation phases, covering methods from prompting to test-time scaling.
Discussion (0). Sign in to comment.