KARPA is a training-free KGQA framework in which an LLM pre-plans relation paths, an embedding model matches them to the knowledge graph, and the LLM reasons over the retrieved paths.
Design of Chain-of-Thought in Math Problem Solving
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Chain-of-Thought (CoT) plays a crucial role in reasoning for math problem solving. We conduct a comprehensive examination of methods for designing CoT, comparing conventional natural language CoT with various program CoTs, including the self-describing program, the comment-describing program, and the non-describing program. Furthermore, we investigate the impact of programming language on program CoTs, comparing Python and Wolfram Language. Through extensive experiments on GSM8K, MATHQA, and SVAMP, we find that program CoTs often have superior effectiveness in math problem solving. Notably, the best performing combination with 30B parameters beats GPT-3.5-turbo by a significant margin. The results show that self-describing program offers greater diversity and thus can generally achieve higher performance. We also find that Python is a better choice of language than Wolfram for program CoTs. The experimental results provide a valuable guideline for future CoT designs that take into account both programming language and coding style for further advancements. Our datasets and code are publicly available.
fields
cs.CL 1years
2024 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
KARPA: A Training-free Method of Adapting Knowledge Graph as References for Large Language Model's Reasoning Path Aggregation
KARPA is a training-free KGQA framework in which an LLM pre-plans relation paths, an embedding model matches them to the knowledge graph, and the LLM reasons over the retrieved paths.