REVIEW 5 major objections 4 minor 1 cited by
Recursive Decomposition of Logical Thoughts: Framework for Superior Reasoning and Knowledge Propagation in Large Language Models
T0 review · 5 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read The paper claims that a prompt-only framework, RDoLT, improves LLM reasoning by recursively decomposing tasks, scoring each thought on four features, and propagating both accepted and rejected thoughts to later stages.
desk verdict RDoLT is a plausible combination with solid experimental breadth, but the load-bearing scoring mechanism is never validated and the paper's own appendix shows it can be arbitrary, so the headline claims do not hold. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The Knowledge Propagation Module (KPM) is the paper's central object: it stores both selected ('strong') and rejected ('weak') thoughts at each decomposition level and makes them available to all later levels, with regeneration when every thought is rejected. Selection is driven by a four-feature thought score, logical validity, coherence, simplicity, and adaptiveness, summed and compared against a threshold $\tau$.
What would settle it
Take a benchmark with known correct answers, run RDoLT, and compare accuracy under the real scorer against accuracy under random thought selection or selection of the lowest-scoring thought; if the real scorer does not beat both, the scoring mechanism is not carrying the reported gain.
Extended reading notes
Core claim
The paper's central claim is that recursive, score-guided decomposition plus memory of rejected thoughts is enough to improve LLM reasoning over existing prompting methods. On GSM8K the reported RDoLT result is 90.98% with ChatGPT-4o, which the paper describes as surpassing the state of the art by 6.28 percentage points; the strongest listed baseline in its comparison table, CoT-SC, is 89.4%. The paper also reports RDoLT as the top method in 65% of the evaluated benchmark settings, with accuracy gains of 5.5 to 6.75 points on other benchmarks. The framework's distinctive mechanism is not a new model but a workflow: generate thoughts, score them on four features, select those above a threshold, and keep weak thoughts visible to later stages for re-evaluation.
Load-bearing premise
Every downstream step consumes the model's own four-feature thought scores, so the framework stands or falls on whether those scores reliably pick thoughts that lead to correct answers.
Editorial extensions
If this is right
- Prompt-only deployment: if RDoLT's numbers hold, users can get 1.5 to 6.75 point accuracy gains on math and letter-concatenation tasks by changing the prompt, with no gradient updates.
- Weak-thought memory matters: KPM's use of rejected thoughts implies that discarded ideas can later become relevant, so methods that discard non-majority reasoning paths may be leaving accuracy on the table.
- Threshold sensitivity is a practical lever: the variant study shows optimal thresholds vary across RDoLT variants, with most peaking at 30 or 35 and all dropping at 40.
- The gain is consistent but not universal: in some benchmark-model cells, such as Gemma-2 27B on MultiArith and Gaokao 2023 Math, CoT remains the better performer in the paper's tables.
Reading between the lines
- Editorial inference: the strongest test of RDoLT is the reliability of its self-scorer; the appendix's MMLU example shows a highest-scoring final thought being selected while the mathematically correct answer appears in a rejected thought, so a direct check of scorer accuracy on known-answer problems would settle whether selection is helping.
- Editorial inference: if the gains are real but threshold-dependent, the method could be made more robust by deriving the threshold from model confidence or adding a second scoring pass, rather than fixing it a priori.
- Editorial inference: KPM's weak-thought memory resembles a short-term associative memory, so applying it to multi-step planning or code generation, where an early wrong branch can be revisited, is a natural next experiment the paper does not run.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces RDoLT, a prompting framework that recursively decomposes a reasoning task into easy, intermediate, and final sub-tasks; generates multiple candidate thoughts per sub-task; scores each thought along four features (Logical Validity, Coherence, Simplicity, Adaptiveness); and propagates both selected and rejected thoughts through a Knowledge Propagation Module (KPM) to later stages. The authors evaluate RDoLT against Vanilla, CoT, CoT-SC, Least-to-Most, and Auto-CoT on GSM8K, SVAMP, MultiArith, LastLetterConcatenation, and Gaokao 2023 Math across five LLMs, claiming consistent superiority and a 6.28% gain over state-of-the-art on GSM8K with ChatGPT-4o. They also report variant analyses with different thresholds and different numbers of thoughts per step.
Significance. If the central claims held, the paper would offer a useful prompt-only method for improving LLM reasoning without fine-tuning. The framework's emphasis on tracking rejected thoughts and explicitly scoring intermediate steps is a reasonable research direction. However, the manuscript's own data contradict the headline claims: several cells in Table 1 show CoT or CoT-SC outperforming RDoLT, and the abstract's 6.28% gain is calculated against the Vanilla baseline rather than against the strongest prior method. The scoring mechanism is unvalidated and is directly contradicted by the Appendix D worked example, and the threshold and thought-count parameters appear to be selected post hoc. The paper does provide detailed worked examples and a reproducible evaluation setup, which is a strength, but the current evidence is insufficient to support the claimed consistent improvement.
major comments (5)
- [Abstract; Table 1] The claim that RDoLT surpasses state-of-the-art techniques by 6.28% on GSM8K with ChatGPT-4o is not supported by Table 1. The 6.28% gap is between RDoLT (90.98) and Vanilla (84.7), whereas the strongest baseline in that row is CoT-SC (89.4), giving a 1.58% advantage. Moreover, Table 1 contains multiple rows where CoT or CoT-SC beat RDoLT: GSM8K Llama3 (CoT-SC 72.86 vs RDoLT 72.63), SVAMP Llama3 (CoT 69.54 vs RDoLT 69.23), SVAMP Gemma2-9B (Auto-CoT 64.52 vs RDoLT 64.19), MultiArith Gemma2-27B (CoT 72.73 vs RDoLT 72.49), and Gaokao Gemma2-27B (CoT 70.28 vs RDoLT 70.05). The statement that RDoLT outperforms other prompting methods in 65% of evaluated benchmarks is also inconsistent with the table, since RDoLT wins roughly 19 of 25 model-benchmark cells; the paper needs a precise aggregate metric with a defined counting rule.
- [§3.3, Eqs. (6)-(10); Appendix D] The four-feature self-scoring system is the mechanism that selects all propagated thoughts, yet the paper never validates that these scores track reasoning quality. Appendix D is a direct counterexample: in the Easy step, Thought 1 receives LV=10 while asserting that (1,2,5,4)(2,3) is a product of disjoint cycles, even though the two cycles share the element 2 and are not disjoint; in the Intermediate step, Thought 2 and Thought 3 both state the same computation 120/4=30 but receive scores of 21 and 36, respectively; and in the Final step, the highest-scoring thought (T3, score 39) produces the wrong answer 2, while the correct thought (T1, score 29) is rejected. Since Eq. (11) selects thoughts based on these scores and Eq. (12) propagates them through KPM, the central selection and propagation mechanism is unsupported.
- [§5.1, Table 2] The robustness analysis sweeps the selection threshold τ from 25 to 40 and shows that the optimal threshold varies across variants (e.g., Single-Step peaks at ≥35, while One-Shot and Multi-Requests variants peak at ≥30). No held-out split, cross-validation procedure, or prespecified rule is given for choosing τ, and it is unclear how the thresholds used to produce Table 1 were fixed. Without such a protocol, the reported accuracies may reflect post-hoc selection of a free parameter that is load-bearing for the KPM selection rule in Eq. (13).
- [§5.2, Table 3] The thought-quantity analysis is internally inconsistent and cannot justify the n=3 setting used in Table 1. The text states that generating five thoughts per step yields a 60.00% success rate, but Table 3 reports 49.75% for that same row; the 'Total Solved' values in the per-step rows (20+30+40=90) match the step columns, yet the reported success rates do not agree with any clearly defined denominator, and the denominator is never stated. The seven-thoughts row similarly reports 77 total solved and a success rate of 38.89, which is inconsistent with 77/210=36.67. Because the paper uses n=3 for all headline results, the choice of n needs a principled justification, not a flawed aggregate table.
- [§5, Table 1] No statistical significance testing, confidence intervals, or multiple-run variance is reported; every comparison is a single point estimate. This is particularly concerning because many RDoLT advantages are below one percentage point (e.g., GSM8K Gemma2-27B: RDoLT 76.58 vs CoT-SC 76.72; MultiArith Gemma2-27B: RDoLT 72.49 vs CoT 72.73). With such small margins and no indication of run-to-run variability or evaluation procedure, the claimed consistent superiority cannot be established.
minor comments (4)
- [§3.2, Eq. (5); §3.3, Eq. (6)] The symbol I is used for the prompt context in Eq. (5) and as an indicator function in Eq. (6); this overloaded notation should be disambiguated.
- [§4.1; Appendix C] There are several typos in dataset names: 'GSMK8' should be 'GSM8K', 'Multi-Arithmetic' should be 'MultiArith', and the appendix heading 'gmsk8' should be capitalized consistently.
- [§4.2; Table 1] Abbreviations are inconsistent: 'Least2Most' is sometimes 'L2M', 'Auto CoT' is sometimes 'A-CoT', and 'CoT-SC' appears as both 'CoT-SC' and 'CoT-SC' without a single definition; the table should use one abbreviation consistently in captions and text.
- [Appendix B] In the KPM block after the Intermediate step, the 'Selected Thoughts (Easy)' line is repeated, but the actual intermediate selected thoughts are not clearly separated from the easy ones; the example should list the intermediate selected thoughts explicitly.
Circularity Check
RDoLT's headline gains are partly fitted: the threshold tau and thought count n are chosen after sweeping on the evaluated benchmarks, and the claimed four-feature scorer is implemented as unvalidated LLM self-scoring.
-
fitted input called prediction
[Section 5.1, Table 2, with Eq. (10)-(11) in Section 3.3]
"RDoLT variants across different threshold score levels revealed intriguing patterns, each with potential implications for practical applications. The single-step (sequential) variant demonstrated the highest overall performance, peaking at 80.78% with a threshold of ≥ 35."
Section 3.3 leaves the selection threshold as a 'predefined threshold tau' without fixing its value. Section 5.1 then sweeps tau in {25, 30, 35, 40} and reports the peaks as the variant's performance. No held-out split or validation set is stated. If the same benchmark family used for the sweep is then reported in Table 1, the headline 90.98% GSM8K figure is the maximum over swept threshold values rather than an out-of-sample prediction. The threshold is thus a fitted input, and the claimed 6.28% gain over SOTA is partly a post-hoc selection artifact rather than a prediction from a fixed configuration.
-
fitted input called prediction
[Section 5.2, Table 3, with Section 3.2 (n=3)]
"The 3 Thoughts Count/Step configuration is identified as the best-performing system."
Section 3.2 fixes n=3 as the number of thoughts generated per level. Section 5.2 then compares n=3, 5, and 7 and, after measuring success rates, declares n=3 the best-performing configuration. The main results, including the 90.98% GSM8K number, inherit this post-hoc choice. Since the selection among thought counts is made from the same benchmark measurements, the reported performance of RDoLT is not the performance of a configuration specified in advance; it is the best of several configurations evaluated on the test data, making part of the reported accuracy a fitted value.
full rationale
This is an empirical prompting-method paper, not a formal derivation, so most circularity patterns are absent: there is no load-bearing self-citation chain (the paper has no obvious self-citations that carry the argument), no imported uniqueness theorem, and no renaming of a known result as a derivation. The central circularity is narrower but concrete: the two free hyperparameters of the method, the selection threshold tau and the number of thoughts n, are chosen after inspecting benchmark results. Section 5.1 sweeps tau and reports peaks; Section 5.2 compares n=3,5,7 and labels 3 as best. If these same benchmark results are then summarized as 'RDoLT consistently outperforms SOTA,' the headline accuracy is a selected maximum over configurations, not an independent prediction, so a central part of the reported gain reduces by construction to fitting. A separate validity issue, which I do not count as circularity, is that the four-feature scoring system defined in Eqs. (6)-(9) is replaced in implementation by 'we let LLM score the thought' (Section 3.3), and Appendix D shows that scorer can assign LV=10 to the mathematically false claim that (1,2,5,4)(2,3) is a product of disjoint cycles while rejecting the correct computation. That is a serious correctness risk for the mechanism, but it is a lack of validation and an internal inconsistency rather than a reduction of the output to the input. Overall score is 6 because the reported performance is partially fitted by threshold and thought-count selection; it is not 8 or 10 because the decomposition idea, the KPM mechanism, and the empirical comparisons are independently describable and could in principle be evaluated with a pre-registered fixed tau and n.
Assumptions & free parameters
free parameters (4)
- number_of_thoughts_n =
3
- selection_threshold_tau =
Not stated for main results; variant peaks at 30 or 35
- scoring_rubric_weights =
Equal weights for S_valid, S_cohere, S_simple, S_adapt
- complexity_and_similarity_functions =
Unspecified
assumptions (4)
- domain assumption LLM self-evaluation for logical validity, coherence, simplicity, and adaptiveness is reliable enough to select good thoughts.
- domain assumption Decomposing tasks into easy, intermediate, and final subtasks improves performance for the tested benchmarks.
- ad hoc to paper The four scoring features capture reasoning quality.
- ad hoc to paper KPM tracking of rejected thoughts provides a downstream benefit.
Cite this review
Pith. "Pith review of Recursive Decomposition of Logical Thoughts: Framework for Superior Reasoning and Knowledge Propagation in Large Language Models." pith.science (2026). https://pith.science/paper/ACNRNWVU
@misc{pith2026250102026,
author = {Pith},
title = {Pith review of: Recursive Decomposition of Logical Thoughts: Framework for Superior Reasoning and Knowledge Propagation in Large Language Models},
year = {2026},
howpublished = {\url{https://pith.science/paper/ACNRNWVU}},
note = {Machine review of arXiv:2501.02026}
}
read the original abstract
Enhancing the reasoning capabilities of Large Language Models remains a critical challenge in artificial intelligence. We introduce RDoLT, Recursive Decomposition of Logical Thought prompting, a novel framework that significantly boosts LLM reasoning performance. RDoLT is built on three key innovations: (1) recursively breaking down complex reasoning tasks into sub-tasks of progressive complexity; (2) employing an advanced selection and scoring mechanism to identify the most promising reasoning thoughts; and (3) integrating a knowledge propagation module that mimics human learning by keeping track of strong and weak thoughts for information propagation. Our approach was evaluated across multiple benchmarks, including GSM8K, SVAMP, MultiArith, LastLetterConcatenation, and Gaokao2023 Math. The results demonstrate that RDoLT consistently outperforms existing state-of-the-art techniques, achieving a 90.98 percent accuracy on GSM8K with ChatGPT-4, surpassing state-of-the-art techniques by 6.28 percent. Similar improvements were observed on other benchmarks, with accuracy gains ranging from 5.5 percent to 6.75 percent. These findings highlight RDoLT's potential to advance prompt engineering, offering a more effective and generalizable approach to complex reasoning tasks.
Figures
Forward citations
Cited by 1 Pith paper
-
MARBLE: A Multi-Agent Rule-Based LLM Reasoning Engine for Accident Severity Prediction
MARBLE claims near-90% accuracy for accident severity prediction by combining a machine learning model with specialized small language model agents and rule-based coordination, though the comparison to baselines is suspect.
Reference graph
Works this paper leans on
-
[1]
Kitab: Evaluating llms on constraint satisfaction for information retrieval. 2023
2023
-
[2]
Information Retrieval meets Large Language Models: A strategic report from Chinese IR community. 2023
2023
-
[3]
Llama 3 Model Card. 2024
2024
-
[4]
Qwen2 Technical Report. 2024
2024
-
[5]
Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. 2022
2022
-
[6]
Graph of Thoughts: Solving Elaborate Problems with Large Language Models. 2024
2024
-
[7]
Topologies of Reasoning: Demystifying Chains, Trees, and Graphs of Thoughts. 2024
2024
-
[8]
Cross-Data Knowledge Graph Construction for LLM-enabled Educational Question-Answering System: A Case Study at HCMUT. 2024
2024
Show all 132 references
-
[9]
INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection. 2024
2024
-
[10]
Teaching Large Language Models to Self-Debug. 2023
2023
-
[11]
ChilleD/LastLetterConcat · Datasets at Hugging Face. 2023
2023
-
[12]
ChilleD/MultiArith · Datasets at Hugging Face. 2023
2023
-
[13]
Training Verifiers to Solve Math Word Problems. 2021
2021
-
[14]
Integrating action knowledge and LLMs for task planning and situation handling in open worlds. 2023
2023
-
[15]
NL-EDIT: Correcting semantic parse errors through natural language interaction. 2021
2021
-
[16]
Neural natural language generation: A survey on multilinguality, multimodality, controllability and learning. 2022
2022
-
[17]
LLMs-based machine translation for E-commerce. 2024
2024
-
[18]
Large Language Models Are Not Strong Abstract Reasoners. 2023
2023
-
[19]
Llms and ai: Understanding its reach and impact. 2023
2023
-
[20]
Abstraction-of-Thought Makes Language Models Better Reasoners. 2024
2024
-
[21]
Large Language Models Cannot Self-Correct Reasoning Yet. 2023
2023
-
[22]
CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning. 2022
2022
-
[23]
MEDIQ: Question-Asking LLMs for Adaptive and Reliable Medical Reasoning. 2024
2024
-
[24]
Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering. 2022
2022
-
[25]
Make llm a testing expert: Bringing human-like interaction to mobile gui testing via functionality-aware decisions. 2024
2024
-
[26]
QUARK: Controllable Text Generation with Reinforced Unlearning. 2022
2022
-
[27]
Self-Refine: Iterative Refinement with Self-Feedback. 2023
2023
-
[28]
LLM-Powered Conversational Voice Assistants: Interaction Patterns, Opportunities, Challenges, and Design Guidelines. 2023
2023
-
[29]
Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals. 2022
2022
-
[30]
Foundation and large language models: fundamentals, challenges, opportunities, and social impacts. 2024
2024
-
[31]
Ollama: Get up and running with Llama 3.1, Mistral, Gemma 2, and other large language models. 2024
2024
-
[32]
GPT-4 Technical Report. 2024
2024
-
[33]
Can Separators Improve Chain-of-Thought Prompting?. 2024
2024
-
[34]
REFINER: Reasoning Feedback on Intermediate Representations. 2024
2024
-
[35]
Machine Translation with Large Language Models: Prompt Engineering for Persian, English, and Russian Directions. 2024
2024
-
[36]
Can Github issues be solved with Tree Of Thoughts?. 2024
2024
-
[37]
Can LLMs Master Math? Investigating Large Language Models on Math Stack Exchange. 2024
2024
-
[38]
Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models. 2023
2023
-
[39]
Small LLMs Are Weak Tool Learners: A Multi-LLM Agent. 2024
2024
-
[40]
Reflexion: Language Agents with Verbal Reinforcement Learning. 2023
2023
-
[41]
Chain of Thoughtlessness? An Analysis of CoT in Planning. 2024
2024
-
[42]
Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback. 2021
2021
-
[43]
ChatGPT as a commenter to the news: can LLMs generate human-like opinions?. 2023
2023
-
[44]
A hybrid intelligence method for argument mining. 2024
2024
-
[45]
A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Tasks. 2024
2024
-
[46]
Self-Consistency Improves Chain of Thought Reasoning in Language Models. 2022
2022
-
[47]
Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents. 2024
2024
-
[48]
Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. 2022
2022
-
[49]
Generating Sequences by Learning to Self-Correct. 2022
2022
-
[50]
MLDT: Multi-Level Decomposition for Complex Long-Horizon Robotic Task Planning with Open-Source Large Language Model. 2024
2024
-
[51]
Self-Evaluation Guided Beam Search for Reasoning. 2023
2023
-
[52]
Designing heterogeneous llm agents for financial sentiment analysis. 2024
2024
-
[53]
Unlocking the potential: A comprehensive exploration of large language models in natural language processing. 2024
2024
-
[54]
Generative AI in the Era of Transformers: Revolutionizing Natural Language Processing with LLMs. 2024
2024
-
[55]
Large Language Models are Contrastive Reasoners. 2024
2024
-
[56]
Tree of Thoughts: Deliberate Problem Solving with Large Language Models. 2023
2023
-
[57]
ReAct: Synergizing Reasoning and Acting in Language Models. 2023
2023
-
[58]
Leveraging generative AI and large Language models: a Comprehensive Roadmap for Healthcare Integration. 2023
2023
-
[59]
A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models. 2024
2024
-
[60]
Exploring collaboration mechanisms for llm agents: A social psychology view. 2023
2023
-
[61]
Evaluating the Performance of Large Language Models on GAOKAO Benchmark. 2023
2023
-
[62]
Misinforming LLMs: vulnerabilities, challenges and opportunities. 2024
2024
-
[63]
Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. 2022
2022
-
[64]
Kitab: Evaluating llms on constraint satisfaction for information retrieval
Marah I Abdin, Suriya Gunasekar, Varun Chandrasekaran, Jerry Li, Mert Yuksekgonul, Rahee Ghosh Peshawaria, Ranjita Naik, and Besmira Nushi. Kitab: Evaluating llms on constraint satisfaction for information retrieval. arXiv preprint arXiv:2310.15511, 2023
-
[65]
Information Retrieval meets Large Language Models: A strategic report from Chinese IR community
Qingyao Ai, Ting Bai, Zhao Cao, Yi Chang, Jiawei Chen, Zhumin Chen, Zhiyong Cheng, Shoubin Dong, Zhicheng Dou, Fuli Feng, Shen Gao, Jiafeng Guo, Xiangnan He, Yanyan Lan, Chenliang Li, Yiqun Liu, Ziyu Lyu, Weizhi Ma, Jun Ma, Zhaochun Ren, Pengjie Ren, Zhiqiang Wang, Mingwen Wan...
2023
-
[66]
Llama 3 model card
AI@Meta. Llama 3 model card. 2024. URL https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md
2024
-
[67]
Training a helpful and harmless assistant with reinforcement learning from human feedback, 2022
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan H...
2022 arXiv
-
[68]
Graph of thoughts: Solving elaborate problems with large language models
Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. Graph of thoughts: Solving elaborate problems with large language models. Proceedings of the...
2024 arXiv
-
[69]
Topologies of reasoning: Demystifying chains, trees, and graphs of thoughts
Maciej Besta, Florim Memedi, Zhenyu Zhang, Robert Gerstenberger, Guangyuan Piao, Nils Blach, Piotr Nyczyk, Marcin Copik, Grzegorz Kwaśniewski, Jürgen Müller, Lukas Gianinazzi, Ales Kubicek, Hubert Niewiadomski, Aidan O'mahony, Onur Mutlu, and Torsten Hoefler. Topologies of rea...
2024 doi
-
[70]
Cross-data knowledge graph construction for llm-enabled educational question-answering system: A case study at hcmut
Tuan Bui, Oanh Tran, Phuong Nguyen, Bao Ho, Long Nguyen, Thang Bui, and Tho Quan. Cross-data knowledge graph construction for llm-enabled educational question-answering system: A case study at hcmut. In Proceedings of the 1st ACM Workshop on AI-Powered Q&A Systems for Multimed...
2024
-
[71]
Inside: Llms' internal states retain the power of hallucination detection
Chao Chen, Kai Liu, Ze Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and Jieping Ye. Inside: Llms' internal states retain the power of hallucination detection. arXiv preprint arXiv:2402.03744, 2024
2024 arXiv
-
[72]
Teaching large language models to self-debug, 2023
Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug, 2023. URL https://arxiv.org/abs/2304.05128
2023 arXiv
-
[73]
ChilleD/LastLetterConcat · Datasets at Hugging Face , 2023 a
ChilleD. ChilleD/LastLetterConcat · Datasets at Hugging Face , 2023 a . URL https://huggingface.co/datasets/ChilleD/LastLetterConcat
2023
-
[74]
ChilleD/MultiArith · Datasets at Hugging Face , 2023 b
ChilleD. ChilleD/MultiArith · Datasets at Hugging Face , 2023 b . URL https://huggingface.co/datasets/ChilleD/MultiArith
2023
-
[75]
Chilled/lastletterconcat · datasets at hugging face
Chilled and Chilled. Chilled/lastletterconcat · datasets at hugging face. 2023. URL https://huggingface.co/datasets/ChilleD/LastLetterConcat
2023
-
[76]
Training verifiers to solve math word problems
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021
-
[77]
Integrating action knowledge and llms for task planning and situation handling in open worlds
Yan Ding, Xiaohan Zhang, Saeid Amiri, Nieqing Cao, Hao Yang, Andy Kaminski, Chad Esselink, and Shiqi Zhang. Integrating action knowledge and llms for task planning and situation handling in open worlds. Autonomous Robots, 47 0 (8): 0 981--997, 2023
2023
-
[78]
Nl-edit: Correcting semantic parse errors through natural language interaction, 2021
Ahmed Elgohary, Christopher Meek, Matthew Richardson, Adam Fourney, Gonzalo Ramos, and Ahmed Hassan Awadallah. Nl-edit: Correcting semantic parse errors through natural language interaction, 2021. URL https://arxiv.org/abs/2103.14540
2021 arXiv
-
[79]
Neural natural language generation: A survey on multilinguality, multimodality, controllability and learning
Erkut Erdem, Menekse Kuyu, Semih Yagcioglu, Anette Frank, Letitia Parcalabescu, Barbara Plank, Andrii Babii, Oleksii Turuta, Aykut Erdem, Iacer Calixto, et al. Neural natural language generation: A survey on multilinguality, multimodality, controllability and learning. Journal...
2022
-
[80]
Llms-based machine translation for e-commerce
Dehong Gao, Kaidi Chen, Ben Chen, Huangyu Dai, Linbo Jin, Wen Jiang, Wei Ning, Shanqing Yu, Qi Xuan, Xiaoyan Cai, et al. Llms-based machine translation for e-commerce. Expert Systems with Applications, page 125087, 2024
2024
- [81]
-
[82]
Llms and ai: Understanding its reach and impact
Anand Gokul. Llms and ai: Understanding its reach and impact. 2023
2023
- [83]
- [84]
-
[85]
Coderl: Mastering code generation through pretrained models and deep reinforcement learning
Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu Hong Hoi. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural ...
2022
-
[86]
Mediq: Question-asking llms for adaptive and reliable medical reasoning
Shuyue Stella Li, Vidhisha Balachandran, Shangbin Feng, Jonathan Ilgen, Emma Pierson, Pang Wei Koh, and Yulia Tsvetkov. Mediq: Question-asking llms for adaptive and reliable medical reasoning. arXiv preprint arXiv:2406.00922, 2024
2024 arXiv
-
[87]
C ommon G en: A constrained text generation challenge for generative commonsense reasoning
Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi, and Xiang Ren. C ommon G en: A constrained text generation challenge for generative commonsense reasoning. In Trevor Cohn, Yulan He, and Yang Liu, editors, Findings of the Association for ...
2020 doi
-
[88]
Rainier: Reinforced knowledge introspector for commonsense question answering, 2022
Jiacheng Liu, Skyler Hallinan, Ximing Lu, Pengfei He, Sean Welleck, Hannaneh Hajishirzi, and Yejin Choi. Rainier: Reinforced knowledge introspector for commonsense question answering, 2022. URL https://arxiv.org/abs/2210.03078
2022 arXiv
-
[89]
Make llm a testing expert: Bringing human-like interaction to mobile gui testing via functionality-aware decisions
Zhe Liu, Chunyang Chen, Junjie Wang, Mengzhuo Chen, Boyu Wu, Xing Che, Dandan Wang, and Qing Wang. Make llm a testing expert: Bringing human-like interaction to mobile gui testing via functionality-aware decisions. In Proceedings of the IEEE/ACM 46th International Conference o...
2024
-
[90]
Quark: Controllable text generation with reinforced unlearning
Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, and Yejin Choi. Quark: Controllable text generation with reinforced unlearning. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural...
2022
-
[91]
Self-refine: Iterative refinement with self-feedback, 2023
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterativ...
2023 arXiv
-
[92]
Llm-powered conversational voice assistants: Interaction patterns, opportunities, challenges, and design guidelines
Amama Mahmood, Junxiang Wang, Bingsheng Yao, Dakuo Wang, and Chien-Ming Huang. Llm-powered conversational voice assistants: Interaction patterns, opportunities, challenges, and design guidelines. arXiv preprint arXiv:2309.13879, 2023
2023 arXiv
-
[93]
Mathewson, Jaylen Pittman, and Richard Evans
Piotr Mirowski, Kory W. Mathewson, Jaylen Pittman, and Richard Evans. Co-writing screenplays and theatre scripts with language models: An evaluation by industry professionals, 2022. URL https://arxiv.org/abs/2209.14958
2022 arXiv
-
[94]
Foundation and large language models: fundamentals, challenges, opportunities, and social impacts
Devon Myers, Rami Mohawesh, Venkata Ishwarya Chellaboina, Anantha Lakshmi Sathvik, Praveen Venkatesh, Yi-Hui Ho, Hanna Henshaw, Muna Alhawawreh, David Berdik, and Yaser Jararweh. Foundation and large language models: fundamentals, challenges, opportunities, and social impacts....
2024
-
[95]
Ollama: Get up and running with Llama 3.1, Mistral, Gemma 2, and other large language models , 2024
Ollama. Ollama: Get up and running with Llama 3.1, Mistral, Gemma 2, and other large language models , 2024. URL https://github.com/ollama/ollama/tree/main
2024
-
[96]
Gpt-4 technical report, 2024
OpenAI. Gpt-4 technical report, 2024. URL https://arxiv.org/abs/2303.08774
2024 arXiv
- [97]
-
[98]
Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2080--2094...
2021 doi
-
[99]
Refiner: Reasoning feedback on intermediate representations, 2024
Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, and Boi Faltings. Refiner: Reasoning feedback on intermediate representations, 2024. URL https://arxiv.org/abs/2304.01904
2024 arXiv
-
[100]
Machine translation with large language models: Prompt engineering for persian, english, and russian directions
Nooshin Pourkamali and Shler Ebrahim Sharifi. Machine translation with large language models: Prompt engineering for persian, english, and russian directions. 2024. URL https://arxiv.org/abs/2401.08429
2024 arXiv
- [101]
-
[102]
Can llms master math? investigating large language models on math stack exchange
Ankit Satpute, Noah Gie ing, Andr\' e Greiner-Petter, Moritz Schubotz, Olaf Teschke, Akiko Aizawa, and Bela Gipp. Can llms master math? investigating large language models on math stack exchange. In Proceedings of the 47th International ACM SIGIR Conference on Research and Dev...
2024
- [103]
- [104]
-
[105]
Reflexion: Language agents with verbal reinforcement learning, 2023
Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning, 2023. URL https://arxiv.org/abs/2303.11366
2023 arXiv
-
[106]
Chain of thoughtlessness? an analysis of cot in planning, 2024
Kaya Stechly, Karthik Valmeekam, and Subbarao Kambhampati. Chain of thoughtlessness? an analysis of cot in planning, 2024. URL https://arxiv.org/abs/2405.04776
2024 arXiv
-
[107]
Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback
Niket Tandon, Aman Madaan, Peter Clark, and Yiming Yang. Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback . arXiv, 2021. doi:10.48550/arxiv.2112.09737
-
[108]
Gemma Team. Gemma. 2024. doi:10.34740/KAGGLE/M/3301. URL https://www.kaggle.com/m/3301
2024 doi
-
[109]
Chatgpt as a commenter to the news: can llms generate human-like opinions? In Multidisciplinary International Symposium on Disinformation in Open Online Media, pages 160--174
Rayden Tseng, Suzan Verberne, and Peter van der Putten. Chatgpt as a commenter to the news: can llms generate human-like opinions? In Multidisciplinary International Symposium on Disinformation in Open Online Media, pages 160--174. Springer, 2023
2023
-
[110]
A hybrid intelligence method for argument mining
Michiel Van Der Meer, Enrico Liscio, Catholijn Jonker, Aske Plaat, Piek Vossen, and Pradeep Murukannaiah. A hybrid intelligence method for argument mining. Journal of Artificial Intelligence Research, 80: 0 1187--1222, 2024
2024
-
[111]
A survey of prompt engineering methods in large language models for different nlp tasks, 2024
Shubham Vatsal and Harsh Dubey. A survey of prompt engineering methods in large language models for different nlp tasks, 2024. URL https://arxiv.org/abs/2407.12994
2024 arXiv
-
[112]
Self-consistency improves chain of thought reasoning in language models
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In ICLR, 2022. URL http://arxiv.org/abs/2203.11171
2022 arXiv
-
[113]
Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents, 2024
Zihao Wang, Shaofei Cai, Guanzhou Chen, Anji Liu, Xiaojian Ma, and Yitao Liang. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents, 2024. URL https://arxiv.org/abs/2302.01560
2024 arXiv
-
[114]
Chain-of-thought prompting elicits reasoning in large language models
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. arXiv, 2022. doi:10.48550/arxiv.2201.11903
-
[115]
Generating sequences by learning to self-correct, 2022
Sean Welleck, Ximing Lu, Peter West, Faeze Brahman, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. Generating sequences by learning to self-correct, 2022. URL https://arxiv.org/abs/2211.00053
2022 arXiv
-
[116]
Mldt: Multi-level decomposition for complex long-horizon robotic task planning with open-source large language model, 2024
Yike Wu, Jiatao Zhang, Nan Hu, LanLing Tang, Guilin Qi, Jun Shao, Jie Ren, and Wei Song. Mldt: Multi-level decomposition for complex long-horizon robotic task planning with open-source large language model, 2024. URL https://arxiv.org/abs/2403.18760
2024 arXiv
-
[117]
Self-evaluation guided beam search for reasoning, 2023
Yuxi Xie, Kenji Kawaguchi, Yiran Zhao, Xu Zhao, Min-Yen Kan, Junxian He, and Qizhe Xie. Self-evaluation guided beam search for reasoning, 2023. URL https://arxiv.org/abs/2305.00633
2023 arXiv
-
[118]
Designing heterogeneous llm agents for financial sentiment analysis
Frank Xing. Designing heterogeneous llm agents for financial sentiment analysis. ACM Transactions on Management Information Systems, 2024
2024
-
[119]
Unlocking the potential: A comprehensive exploration of large language models in natural language processing
Qing Xue. Unlocking the potential: A comprehensive exploration of large language models in natural language processing. Applied and Computational Engineering, 57 0 (1): 0 247--252, 2024. ISSN 2755-2721. doi:10.54254/2755-2721/57/20241341
2024 doi
-
[120]
Generative ai in the era of transformers: Revolutionizing natural language processing with llms
Archna Balkrishna Yadav. Generative ai in the era of transformers: Revolutionizing natural language processing with llms. Journal of Image Processing and Intelligent Remote Sensing, 0 (42): 0 54--61, 2024. ISSN 2815-0953. doi:10.55529/jipirs.42.54.61. URL https://pdfs.semantic...
2024 doi
-
[121]
Qwen2 technical report, 2024
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze...
2024 arXiv
-
[122]
Large language models are contrastive reasoners
Liang Yao. Large language models are contrastive reasoners. arXiv, 2024. doi:10.48550/arxiv.2403.08211
2024 doi
-
[123]
Griffiths, Yuan Cao, and Karthik Narasimhan
Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36, 2023 a . ISSN 10495258. URL https://arxiv.or...
2023 arXiv
-
[124]
React: Synergizing reasoning and acting in language models, 2023 b
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models, 2023 b . URL https://arxiv.org/abs/2210.03629
2023 arXiv
-
[125]
Leveraging generative ai and large language models: a comprehensive roadmap for healthcare integration
Ping Yu, Hua Xu, Xia Hu, and Chao Deng. Leveraging generative ai and large language models: a comprehensive roadmap for healthcare integration. In Healthcare, volume 11, page 2776. MDPI, 2023
2023
-
[126]
A systematic survey of text summarization: From statistical methods to large language models
Haopeng Zhang, Philip S Yu, and Jiawei Zhang. A systematic survey of text summarization: From statistical methods to large language models. arXiv preprint arXiv:2406.11289, 2024
2024 arXiv
-
[127]
Exploring collaboration mechanisms for llm agents: A social psychology view
Jintian Zhang, Xin Xu, and Shumin Deng. Exploring collaboration mechanisms for llm agents: A social psychology view. arXiv preprint arXiv:2310.02124, 2023 a
2023 arXiv
-
[128]
Evaluating the performance of large language models on gaokao benchmark
Xiaotian Zhang, Chunyang Li, Yi Zong, Zhengyu Ying, Liang He, and Xipeng Qiu. Evaluating the performance of large language models on gaokao benchmark. 2023 b
2023
-
[129]
Misinforming llms: vulnerabilities, challenges and opportunities, 2024
Bo Zhou, Daniel Geißler, and Paul Lukowicz. Misinforming llms: vulnerabilities, challenges and opportunities, 2024. URL https://arxiv.org/abs/2408.01168
2024 arXiv
-
[130]
Least-to-most prompting enables complex reasoning in large language models
Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and Ed Chi. Least-to-most prompting enables complex reasoning in large language models. The International Conference on Learning Representation...
2022 arXiv
-
[131]
write newline
" write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION write.doi doi empty "" new.block "DOI: " doi * "" * if FUNCTION write.url doi empty url empty ...
-
[132]
write newline
" write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.