Pith. sign in

REVIEW 5 major objections 4 minor 1 cited by

Recursive Decomposition of Logical Thoughts: Framework for Superior Reasoning and Knowledge Propagation in Large Language Models

T0 review · 5 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Pith's one-line read The paper claims that a prompt-only framework, RDoLT, improves LLM reasoning by recursively decomposing tasks, scoring each thought on four features, and propagating both accepted and rejected thoughts to later stages.

desk verdict RDoLT is a plausible combination with solid experimental breadth, but the load-bearing scoring mechanism is never validated and the paper's own appendix shows it can be arbitrary, so the headline claims do not hold. read the letter →

arxiv 2501.02026 v1 pith:ACNRNWVU submitted 2025-01-03 cs.CL cs.AIcs.LGcs.LO

classification cs.CLcs.AIcs.LGcs.LO
keywords promptengineeringrecursivedecompositionchain-of-thoughtpromptingknowledgepropagationthoughtscoringlargelanguagemodelsmathematicalreasoning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

RDoLT is a prompting-only framework that tries to make large language models reason better without fine-tuning. It decomposes a problem into easy, intermediate, and final sub-tasks, generates several candidate thoughts at each stage, scores them on logical validity, coherence, simplicity, and adaptiveness, and then propagates both accepted and rejected thoughts forward through a Knowledge Propagation Module. The paper claims this consistently beats chain-of-thought, self-consistency, least-to-most, and auto-CoT prompting across five benchmarks, including a 90.98% GSM8K accuracy with ChatGPT-4o. If true, a purely textual change to the prompt would yield robust accuracy gains of roughly 1.5 to 6.75 points across models and task families.

What carries the argument

The Knowledge Propagation Module (KPM) is the paper's central object: it stores both selected ('strong') and rejected ('weak') thoughts at each decomposition level and makes them available to all later levels, with regeneration when every thought is rejected. Selection is driven by a four-feature thought score, logical validity, coherence, simplicity, and adaptiveness, summed and compared against a threshold $\tau$.

What would settle it

Take a benchmark with known correct answers, run RDoLT, and compare accuracy under the real scorer against accuracy under random thought selection or selection of the lowest-scoring thought; if the real scorer does not beat both, the scoring mechanism is not carrying the reported gain.

Watch

Extended reading notes

Core claim

The paper's central claim is that recursive, score-guided decomposition plus memory of rejected thoughts is enough to improve LLM reasoning over existing prompting methods. On GSM8K the reported RDoLT result is 90.98% with ChatGPT-4o, which the paper describes as surpassing the state of the art by 6.28 percentage points; the strongest listed baseline in its comparison table, CoT-SC, is 89.4%. The paper also reports RDoLT as the top method in 65% of the evaluated benchmark settings, with accuracy gains of 5.5 to 6.75 points on other benchmarks. The framework's distinctive mechanism is not a new model but a workflow: generate thoughts, score them on four features, select those above a threshold, and keep weak thoughts visible to later stages for re-evaluation.

Load-bearing premise

Every downstream step consumes the model's own four-feature thought scores, so the framework stands or falls on whether those scores reliably pick thoughts that lead to correct answers.

Editorial extensions

If this is right

  • Prompt-only deployment: if RDoLT's numbers hold, users can get 1.5 to 6.75 point accuracy gains on math and letter-concatenation tasks by changing the prompt, with no gradient updates.
  • Weak-thought memory matters: KPM's use of rejected thoughts implies that discarded ideas can later become relevant, so methods that discard non-majority reasoning paths may be leaving accuracy on the table.
  • Threshold sensitivity is a practical lever: the variant study shows optimal thresholds vary across RDoLT variants, with most peaking at 30 or 35 and all dropping at 40.
  • The gain is consistent but not universal: in some benchmark-model cells, such as Gemma-2 27B on MultiArith and Gaokao 2023 Math, CoT remains the better performer in the paper's tables.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the strongest test of RDoLT is the reliability of its self-scorer; the appendix's MMLU example shows a highest-scoring final thought being selected while the mathematically correct answer appears in a rejected thought, so a direct check of scorer accuracy on known-answer problems would settle whether selection is helping.
  • Editorial inference: if the gains are real but threshold-dependent, the method could be made more robust by deriving the threshold from model confidence or adding a second scoring pass, rather than fixing it a priori.
  • Editorial inference: KPM's weak-thought memory resembles a short-term associative memory, so applying it to multi-step planning or code generation, where an early wrong branch can be revisited, is a natural next experiment the paper does not run.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 4 minor

Summary. The manuscript introduces RDoLT, a prompting framework that recursively decomposes a reasoning task into easy, intermediate, and final sub-tasks; generates multiple candidate thoughts per sub-task; scores each thought along four features (Logical Validity, Coherence, Simplicity, Adaptiveness); and propagates both selected and rejected thoughts through a Knowledge Propagation Module (KPM) to later stages. The authors evaluate RDoLT against Vanilla, CoT, CoT-SC, Least-to-Most, and Auto-CoT on GSM8K, SVAMP, MultiArith, LastLetterConcatenation, and Gaokao 2023 Math across five LLMs, claiming consistent superiority and a 6.28% gain over state-of-the-art on GSM8K with ChatGPT-4o. They also report variant analyses with different thresholds and different numbers of thoughts per step.

Significance. If the central claims held, the paper would offer a useful prompt-only method for improving LLM reasoning without fine-tuning. The framework's emphasis on tracking rejected thoughts and explicitly scoring intermediate steps is a reasonable research direction. However, the manuscript's own data contradict the headline claims: several cells in Table 1 show CoT or CoT-SC outperforming RDoLT, and the abstract's 6.28% gain is calculated against the Vanilla baseline rather than against the strongest prior method. The scoring mechanism is unvalidated and is directly contradicted by the Appendix D worked example, and the threshold and thought-count parameters appear to be selected post hoc. The paper does provide detailed worked examples and a reproducible evaluation setup, which is a strength, but the current evidence is insufficient to support the claimed consistent improvement.

major comments (5)
  1. [Abstract; Table 1] The claim that RDoLT surpasses state-of-the-art techniques by 6.28% on GSM8K with ChatGPT-4o is not supported by Table 1. The 6.28% gap is between RDoLT (90.98) and Vanilla (84.7), whereas the strongest baseline in that row is CoT-SC (89.4), giving a 1.58% advantage. Moreover, Table 1 contains multiple rows where CoT or CoT-SC beat RDoLT: GSM8K Llama3 (CoT-SC 72.86 vs RDoLT 72.63), SVAMP Llama3 (CoT 69.54 vs RDoLT 69.23), SVAMP Gemma2-9B (Auto-CoT 64.52 vs RDoLT 64.19), MultiArith Gemma2-27B (CoT 72.73 vs RDoLT 72.49), and Gaokao Gemma2-27B (CoT 70.28 vs RDoLT 70.05). The statement that RDoLT outperforms other prompting methods in 65% of evaluated benchmarks is also inconsistent with the table, since RDoLT wins roughly 19 of 25 model-benchmark cells; the paper needs a precise aggregate metric with a defined counting rule.
  2. [§3.3, Eqs. (6)-(10); Appendix D] The four-feature self-scoring system is the mechanism that selects all propagated thoughts, yet the paper never validates that these scores track reasoning quality. Appendix D is a direct counterexample: in the Easy step, Thought 1 receives LV=10 while asserting that (1,2,5,4)(2,3) is a product of disjoint cycles, even though the two cycles share the element 2 and are not disjoint; in the Intermediate step, Thought 2 and Thought 3 both state the same computation 120/4=30 but receive scores of 21 and 36, respectively; and in the Final step, the highest-scoring thought (T3, score 39) produces the wrong answer 2, while the correct thought (T1, score 29) is rejected. Since Eq. (11) selects thoughts based on these scores and Eq. (12) propagates them through KPM, the central selection and propagation mechanism is unsupported.
  3. [§5.1, Table 2] The robustness analysis sweeps the selection threshold τ from 25 to 40 and shows that the optimal threshold varies across variants (e.g., Single-Step peaks at ≥35, while One-Shot and Multi-Requests variants peak at ≥30). No held-out split, cross-validation procedure, or prespecified rule is given for choosing τ, and it is unclear how the thresholds used to produce Table 1 were fixed. Without such a protocol, the reported accuracies may reflect post-hoc selection of a free parameter that is load-bearing for the KPM selection rule in Eq. (13).
  4. [§5.2, Table 3] The thought-quantity analysis is internally inconsistent and cannot justify the n=3 setting used in Table 1. The text states that generating five thoughts per step yields a 60.00% success rate, but Table 3 reports 49.75% for that same row; the 'Total Solved' values in the per-step rows (20+30+40=90) match the step columns, yet the reported success rates do not agree with any clearly defined denominator, and the denominator is never stated. The seven-thoughts row similarly reports 77 total solved and a success rate of 38.89, which is inconsistent with 77/210=36.67. Because the paper uses n=3 for all headline results, the choice of n needs a principled justification, not a flawed aggregate table.
  5. [§5, Table 1] No statistical significance testing, confidence intervals, or multiple-run variance is reported; every comparison is a single point estimate. This is particularly concerning because many RDoLT advantages are below one percentage point (e.g., GSM8K Gemma2-27B: RDoLT 76.58 vs CoT-SC 76.72; MultiArith Gemma2-27B: RDoLT 72.49 vs CoT 72.73). With such small margins and no indication of run-to-run variability or evaluation procedure, the claimed consistent superiority cannot be established.
minor comments (4)
  1. [§3.2, Eq. (5); §3.3, Eq. (6)] The symbol I is used for the prompt context in Eq. (5) and as an indicator function in Eq. (6); this overloaded notation should be disambiguated.
  2. [§4.1; Appendix C] There are several typos in dataset names: 'GSMK8' should be 'GSM8K', 'Multi-Arithmetic' should be 'MultiArith', and the appendix heading 'gmsk8' should be capitalized consistently.
  3. [§4.2; Table 1] Abbreviations are inconsistent: 'Least2Most' is sometimes 'L2M', 'Auto CoT' is sometimes 'A-CoT', and 'CoT-SC' appears as both 'CoT-SC' and 'CoT-SC' without a single definition; the table should use one abbreviation consistently in captions and text.
  4. [Appendix B] In the KPM block after the Intermediate step, the 'Selected Thoughts (Easy)' line is repeated, but the actual intermediate selected thoughts are not clearly separated from the easy ones; the example should list the intermediate selected thoughts explicitly.

Circularity Check

2 steps flagged · score 6.0 of 10

RDoLT's headline gains are partly fitted: the threshold tau and thought count n are chosen after sweeping on the evaluated benchmarks, and the claimed four-feature scorer is implemented as unvalidated LLM self-scoring.

  1. fitted input called prediction [Section 5.1, Table 2, with Eq. (10)-(11) in Section 3.3]
    "RDoLT variants across different threshold score levels revealed intriguing patterns, each with potential implications for practical applications. The single-step (sequential) variant demonstrated the highest overall performance, peaking at 80.78% with a threshold of ≥ 35."

    Section 3.3 leaves the selection threshold as a 'predefined threshold tau' without fixing its value. Section 5.1 then sweeps tau in {25, 30, 35, 40} and reports the peaks as the variant's performance. No held-out split or validation set is stated. If the same benchmark family used for the sweep is then reported in Table 1, the headline 90.98% GSM8K figure is the maximum over swept threshold values rather than an out-of-sample prediction. The threshold is thus a fitted input, and the claimed 6.28% gain over SOTA is partly a post-hoc selection artifact rather than a prediction from a fixed configuration.

  2. fitted input called prediction [Section 5.2, Table 3, with Section 3.2 (n=3)]
    "The 3 Thoughts Count/Step configuration is identified as the best-performing system."

    Section 3.2 fixes n=3 as the number of thoughts generated per level. Section 5.2 then compares n=3, 5, and 7 and, after measuring success rates, declares n=3 the best-performing configuration. The main results, including the 90.98% GSM8K number, inherit this post-hoc choice. Since the selection among thought counts is made from the same benchmark measurements, the reported performance of RDoLT is not the performance of a configuration specified in advance; it is the best of several configurations evaluated on the test data, making part of the reported accuracy a fitted value.

full rationale

This is an empirical prompting-method paper, not a formal derivation, so most circularity patterns are absent: there is no load-bearing self-citation chain (the paper has no obvious self-citations that carry the argument), no imported uniqueness theorem, and no renaming of a known result as a derivation. The central circularity is narrower but concrete: the two free hyperparameters of the method, the selection threshold tau and the number of thoughts n, are chosen after inspecting benchmark results. Section 5.1 sweeps tau and reports peaks; Section 5.2 compares n=3,5,7 and labels 3 as best. If these same benchmark results are then summarized as 'RDoLT consistently outperforms SOTA,' the headline accuracy is a selected maximum over configurations, not an independent prediction, so a central part of the reported gain reduces by construction to fitting. A separate validity issue, which I do not count as circularity, is that the four-feature scoring system defined in Eqs. (6)-(9) is replaced in implementation by 'we let LLM score the thought' (Section 3.3), and Appendix D shows that scorer can assign LV=10 to the mathematically false claim that (1,2,5,4)(2,3) is a product of disjoint cycles while rejecting the correct computation. That is a serious correctness risk for the mechanism, but it is a lack of validation and an internal inconsistency rather than a reduction of the output to the input. Overall score is 6 because the reported performance is partially fitted by threshold and thought-count selection; it is not 8 or 10 because the decomposition idea, the KPM mechanism, and the empirical comparisons are independently describable and could in principle be evaluated with a pre-registered fixed tau and n.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The framework rests on self-assessment and a threshold parameter that is tuned on the test benchmarks; neither is given independent validation. The method is modular, but the module-level claims are untested by ablation.

free parameters (4)
  • number_of_thoughts_n = 3
    Set to three in §3.2; Table 3 suggests performance varies with n, so the choice affects reported accuracy.
  • selection_threshold_tau = Not stated for main results; variant peaks at 30 or 35
    The threshold is a free parameter explored in Table 2. The paper does not describe a held-out validation procedure, so the main table numbers may implicitly depend on threshold choice.
  • scoring_rubric_weights = Equal weights for S_valid, S_cohere, S_simple, S_adapt
    Equation 10 sums four features with equal weight; no justification or sensitivity analysis is given.
  • complexity_and_similarity_functions = Unspecified
    The paper says complexity could be length or step count, and similarity uses cosine similarity or LLM judgment; both are underspecified and effectively chosen by the experimenter.
assumptions (4)
  • domain assumption LLM self-evaluation for logical validity, coherence, simplicity, and adaptiveness is reliable enough to select good thoughts.
    Invoked when RDoLT lets the LLM score thoughts (§3.3) and uses these scores to propagate thoughts (§3.4). Contradicted by the MMLU appendix example where the wrong thought receives the highest score.
  • domain assumption Decomposing tasks into easy, intermediate, and final subtasks improves performance for the tested benchmarks.
    The method's central structure (§3.1); no ablation shows that a two-stage or flat decomposition performs worse.
  • ad hoc to paper The four scoring features capture reasoning quality.
    Equations 6-9 are introduced for this paper without empirical validation.
  • ad hoc to paper KPM tracking of rejected thoughts provides a downstream benefit.
    The key claimed innovation (§3.4) is never ablated, so its contribution is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Recursive Decomposition of Logical Thoughts: Framework for Superior Reasoning and Knowledge Propagation in Large Language Models." pith.science (2026). https://pith.science/paper/ACNRNWVU

@misc{pith2026250102026,
  author       = {Pith},
  title        = {Pith review of: Recursive Decomposition of Logical Thoughts: Framework for Superior Reasoning and Knowledge Propagation in Large Language Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ACNRNWVU}},
  note         = {Machine review of arXiv:2501.02026}
}
read the original abstract

Enhancing the reasoning capabilities of Large Language Models remains a critical challenge in artificial intelligence. We introduce RDoLT, Recursive Decomposition of Logical Thought prompting, a novel framework that significantly boosts LLM reasoning performance. RDoLT is built on three key innovations: (1) recursively breaking down complex reasoning tasks into sub-tasks of progressive complexity; (2) employing an advanced selection and scoring mechanism to identify the most promising reasoning thoughts; and (3) integrating a knowledge propagation module that mimics human learning by keeping track of strong and weak thoughts for information propagation. Our approach was evaluated across multiple benchmarks, including GSM8K, SVAMP, MultiArith, LastLetterConcatenation, and Gaokao2023 Math. The results demonstrate that RDoLT consistently outperforms existing state-of-the-art techniques, achieving a 90.98 percent accuracy on GSM8K with ChatGPT-4, surpassing state-of-the-art techniques by 6.28 percent. Similar improvements were observed on other benchmarks, with accuracy gains ranging from 5.5 percent to 6.75 percent. These findings highlight RDoLT's potential to advance prompt engineering, offering a more effective and generalizable approach to complex reasoning tasks.

Figures

Figures reproduced from arXiv: 2501.02026 by the authors.

Figure 1
Figure 1. Illustration of Recursive Decomposition of Logical Thoughts (RDoLT) Framework. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Edge cases handled by the Knowledge Propagation Module (KPM) during thought [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Detailed example illustrating how RDoLT addresses problems by [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Impact of different thought selection strategies on accuracy using KPM. Strategies [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. MARBLE: A Multi-Agent Rule-Based LLM Reasoning Engine for Accident Severity Prediction

    cs.AI 2025-07 reject novelty 4.0 of 10

    MARBLE claims near-90% accuracy for accident severity prediction by combining a machine learning model with specialized small language model agents and rule-based coordination, though the comparison to baselines is suspect.

Reference graph

Works this paper leans on

132 extracted references · 37 canonical work pages · cited by 1 Pith paper

  1. [1]

    Kitab: Evaluating llms on constraint satisfaction for information retrieval. 2023

  2. [2]

    Information Retrieval meets Large Language Models: A strategic report from Chinese IR community. 2023

  3. [3]

    Llama 3 Model Card. 2024

  4. [4]

    Qwen2 Technical Report. 2024

  5. [5]

    Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. 2022

  6. [6]

    Graph of Thoughts: Solving Elaborate Problems with Large Language Models. 2024

  7. [7]

    Topologies of Reasoning: Demystifying Chains, Trees, and Graphs of Thoughts. 2024

  8. [8]

    Cross-Data Knowledge Graph Construction for LLM-enabled Educational Question-Answering System: A Case Study at HCMUT. 2024

Show all 132 references
  1. [9]

    INSIDE: LLMs' Internal States Retain the Power of Hallucination Detection. 2024

  2. [10]

    Teaching Large Language Models to Self-Debug. 2023

  3. [11]

    ChilleD/LastLetterConcat · Datasets at Hugging Face. 2023

  4. [12]

    ChilleD/MultiArith · Datasets at Hugging Face. 2023

  5. [13]

    Training Verifiers to Solve Math Word Problems. 2021

  6. [14]

    Integrating action knowledge and LLMs for task planning and situation handling in open worlds. 2023

  7. [15]

    NL-EDIT: Correcting semantic parse errors through natural language interaction. 2021

  8. [16]

    Neural natural language generation: A survey on multilinguality, multimodality, controllability and learning. 2022

  9. [17]

    LLMs-based machine translation for E-commerce. 2024

  10. [18]

    Large Language Models Are Not Strong Abstract Reasoners. 2023

  11. [19]

    Llms and ai: Understanding its reach and impact. 2023

  12. [20]

    Abstraction-of-Thought Makes Language Models Better Reasoners. 2024

  13. [21]

    Large Language Models Cannot Self-Correct Reasoning Yet. 2023

  14. [22]

    CodeRL: Mastering Code Generation through Pretrained Models and Deep Reinforcement Learning. 2022

  15. [23]

    MEDIQ: Question-Asking LLMs for Adaptive and Reliable Medical Reasoning. 2024

  16. [24]

    Rainier: Reinforced Knowledge Introspector for Commonsense Question Answering. 2022

  17. [25]

    Make llm a testing expert: Bringing human-like interaction to mobile gui testing via functionality-aware decisions. 2024

  18. [26]

    QUARK: Controllable Text Generation with Reinforced Unlearning. 2022

  19. [27]

    Self-Refine: Iterative Refinement with Self-Feedback. 2023

  20. [28]

    LLM-Powered Conversational Voice Assistants: Interaction Patterns, Opportunities, Challenges, and Design Guidelines. 2023

  21. [29]

    Co-Writing Screenplays and Theatre Scripts with Language Models: An Evaluation by Industry Professionals. 2022

  22. [30]

    Foundation and large language models: fundamentals, challenges, opportunities, and social impacts. 2024

  23. [31]

    Ollama: Get up and running with Llama 3.1, Mistral, Gemma 2, and other large language models. 2024

  24. [32]

    GPT-4 Technical Report. 2024

  25. [33]

    Can Separators Improve Chain-of-Thought Prompting?. 2024

  26. [34]

    REFINER: Reasoning Feedback on Intermediate Representations. 2024

  27. [35]

    Machine Translation with Large Language Models: Prompt Engineering for Persian, English, and Russian Directions. 2024

  28. [36]

    Can Github issues be solved with Tree Of Thoughts?. 2024

  29. [37]

    Can LLMs Master Math? Investigating Large Language Models on Math Stack Exchange. 2024

  30. [38]

    Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models. 2023

  31. [39]

    Small LLMs Are Weak Tool Learners: A Multi-LLM Agent. 2024

  32. [40]

    Reflexion: Language Agents with Verbal Reinforcement Learning. 2023

  33. [41]

    Chain of Thoughtlessness? An Analysis of CoT in Planning. 2024

  34. [42]

    Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback. 2021

  35. [43]

    ChatGPT as a commenter to the news: can LLMs generate human-like opinions?. 2023

  36. [44]

    A hybrid intelligence method for argument mining. 2024

  37. [45]

    A Survey of Prompt Engineering Methods in Large Language Models for Different NLP Tasks. 2024

  38. [46]

    Self-Consistency Improves Chain of Thought Reasoning in Language Models. 2022

  39. [47]

    Describe, Explain, Plan and Select: Interactive Planning with Large Language Models Enables Open-World Multi-Task Agents. 2024

  40. [48]

    Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. 2022

  41. [49]

    Generating Sequences by Learning to Self-Correct. 2022

  42. [50]

    MLDT: Multi-Level Decomposition for Complex Long-Horizon Robotic Task Planning with Open-Source Large Language Model. 2024

  43. [51]

    Self-Evaluation Guided Beam Search for Reasoning. 2023

  44. [52]

    Designing heterogeneous llm agents for financial sentiment analysis. 2024

  45. [53]

    Unlocking the potential: A comprehensive exploration of large language models in natural language processing. 2024

  46. [54]

    Generative AI in the Era of Transformers: Revolutionizing Natural Language Processing with LLMs. 2024

  47. [55]

    Large Language Models are Contrastive Reasoners. 2024

  48. [56]

    Tree of Thoughts: Deliberate Problem Solving with Large Language Models. 2023

  49. [57]

    ReAct: Synergizing Reasoning and Acting in Language Models. 2023

  50. [58]

    Leveraging generative AI and large Language models: a Comprehensive Roadmap for Healthcare Integration. 2023

  51. [59]

    A Systematic Survey of Text Summarization: From Statistical Methods to Large Language Models. 2024

  52. [60]

    Exploring collaboration mechanisms for llm agents: A social psychology view. 2023

  53. [61]

    Evaluating the Performance of Large Language Models on GAOKAO Benchmark. 2023

  54. [62]

    Misinforming LLMs: vulnerabilities, challenges and opportunities. 2024

  55. [63]

    Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. 2022

  56. [64]

    Kitab: Evaluating llms on constraint satisfaction for information retrieval

    Marah I Abdin, Suriya Gunasekar, Varun Chandrasekaran, Jerry Li, Mert Yuksekgonul, Rahee Ghosh Peshawaria, Ranjita Naik, and Besmira Nushi. Kitab: Evaluating llms on constraint satisfaction for information retrieval. arXiv preprint arXiv:2310.15511, 2023

  57. [65]

    Information Retrieval meets Large Language Models: A strategic report from Chinese IR community

    Qingyao Ai, Ting Bai, Zhao Cao, Yi Chang, Jiawei Chen, Zhumin Chen, Zhiyong Cheng, Shoubin Dong, Zhicheng Dou, Fuli Feng, Shen Gao, Jiafeng Guo, Xiangnan He, Yanyan Lan, Chenliang Li, Yiqun Liu, Ziyu Lyu, Weizhi Ma, Jun Ma, Zhaochun Ren, Pengjie Ren, Zhiqiang Wang, Mingwen Wan...

  58. [66]

    Llama 3 model card

    AI@Meta. Llama 3 model card. 2024. URL https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md

  59. [67]

    Training a helpful and harmless assistant with reinforcement learning from human feedback, 2022

    Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan H...

  60. [68]

    Graph of thoughts: Solving elaborate problems with large language models

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. Graph of thoughts: Solving elaborate problems with large language models. Proceedings of the...

  61. [69]

    Topologies of reasoning: Demystifying chains, trees, and graphs of thoughts

    Maciej Besta, Florim Memedi, Zhenyu Zhang, Robert Gerstenberger, Guangyuan Piao, Nils Blach, Piotr Nyczyk, Marcin Copik, Grzegorz Kwaśniewski, Jürgen Müller, Lukas Gianinazzi, Ales Kubicek, Hubert Niewiadomski, Aidan O'mahony, Onur Mutlu, and Torsten Hoefler. Topologies of rea...

  62. [70]

    Cross-data knowledge graph construction for llm-enabled educational question-answering system: A case study at hcmut

    Tuan Bui, Oanh Tran, Phuong Nguyen, Bao Ho, Long Nguyen, Thang Bui, and Tho Quan. Cross-data knowledge graph construction for llm-enabled educational question-answering system: A case study at hcmut. In Proceedings of the 1st ACM Workshop on AI-Powered Q&A Systems for Multimed...

  63. [71]

    Inside: Llms' internal states retain the power of hallucination detection

    Chao Chen, Kai Liu, Ze Chen, Yi Gu, Yue Wu, Mingyuan Tao, Zhihang Fu, and Jieping Ye. Inside: Llms' internal states retain the power of hallucination detection. arXiv preprint arXiv:2402.03744, 2024

  64. [72]

    Teaching large language models to self-debug, 2023

    Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug, 2023. URL https://arxiv.org/abs/2304.05128

  65. [73]

    ChilleD/LastLetterConcat · Datasets at Hugging Face , 2023 a

    ChilleD. ChilleD/LastLetterConcat · Datasets at Hugging Face , 2023 a . URL https://huggingface.co/datasets/ChilleD/LastLetterConcat

  66. [74]

    ChilleD/MultiArith · Datasets at Hugging Face , 2023 b

    ChilleD. ChilleD/MultiArith · Datasets at Hugging Face , 2023 b . URL https://huggingface.co/datasets/ChilleD/MultiArith

  67. [75]

    Chilled/lastletterconcat · datasets at hugging face

    Chilled and Chilled. Chilled/lastletterconcat · datasets at hugging face. 2023. URL https://huggingface.co/datasets/ChilleD/LastLetterConcat

  68. [76]

    Training verifiers to solve math word problems

    Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021

  69. [77]

    Integrating action knowledge and llms for task planning and situation handling in open worlds

    Yan Ding, Xiaohan Zhang, Saeid Amiri, Nieqing Cao, Hao Yang, Andy Kaminski, Chad Esselink, and Shiqi Zhang. Integrating action knowledge and llms for task planning and situation handling in open worlds. Autonomous Robots, 47 0 (8): 0 981--997, 2023

  70. [78]

    Nl-edit: Correcting semantic parse errors through natural language interaction, 2021

    Ahmed Elgohary, Christopher Meek, Matthew Richardson, Adam Fourney, Gonzalo Ramos, and Ahmed Hassan Awadallah. Nl-edit: Correcting semantic parse errors through natural language interaction, 2021. URL https://arxiv.org/abs/2103.14540

  71. [79]

    Neural natural language generation: A survey on multilinguality, multimodality, controllability and learning

    Erkut Erdem, Menekse Kuyu, Semih Yagcioglu, Anette Frank, Letitia Parcalabescu, Barbara Plank, Andrii Babii, Oleksii Turuta, Aykut Erdem, Iacer Calixto, et al. Neural natural language generation: A survey on multilinguality, multimodality, controllability and learning. Journal...

  72. [80]

    Llms-based machine translation for e-commerce

    Dehong Gao, Kaidi Chen, Ben Chen, Huangyu Dai, Linbo Jin, Wen Jiang, Wei Ning, Shanqing Yu, Qi Xuan, Xiaoyan Cai, et al. Llms-based machine translation for e-commerce. Expert Systems with Applications, page 125087, 2024

  73. [81]

    Large language models are not strong abstract reasoners

    Gaël Gendron, Qiming Bao, Michael Witbrock, and Gillian Dobbie. Large language models are not strong abstract reasoners. arXiv, 2023. doi:10.48550/arxiv.2305.19555

  74. [82]

    Llms and ai: Understanding its reach and impact

    Anand Gokul. Llms and ai: Understanding its reach and impact. 2023

  75. [83]

    Abstraction-of-thought makes language models better reasoners

    Ruixin Hong, Hongming Zhang, Xiaoman Pan, Dong Yu, and Changshui Zhang. Abstraction-of-thought makes language models better reasoners. arXiv, 2024. doi:10.48550/arxiv.2406.12442

  76. [84]

    Large language models cannot self-correct reasoning yet

    Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. arXiv, 2023. doi:10.48550/arxiv.2310.01798

  77. [85]

    Coderl: Mastering code generation through pretrained models and deep reinforcement learning

    Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu Hong Hoi. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural ...

  78. [86]

    Mediq: Question-asking llms for adaptive and reliable medical reasoning

    Shuyue Stella Li, Vidhisha Balachandran, Shangbin Feng, Jonathan Ilgen, Emma Pierson, Pang Wei Koh, and Yulia Tsvetkov. Mediq: Question-asking llms for adaptive and reliable medical reasoning. arXiv preprint arXiv:2406.00922, 2024

  79. [87]

    C ommon G en: A constrained text generation challenge for generative commonsense reasoning

    Bill Yuchen Lin, Wangchunshu Zhou, Ming Shen, Pei Zhou, Chandra Bhagavatula, Yejin Choi, and Xiang Ren. C ommon G en: A constrained text generation challenge for generative commonsense reasoning. In Trevor Cohn, Yulan He, and Yang Liu, editors, Findings of the Association for ...

  80. [88]

    Rainier: Reinforced knowledge introspector for commonsense question answering, 2022

    Jiacheng Liu, Skyler Hallinan, Ximing Lu, Pengfei He, Sean Welleck, Hannaneh Hajishirzi, and Yejin Choi. Rainier: Reinforced knowledge introspector for commonsense question answering, 2022. URL https://arxiv.org/abs/2210.03078

  81. [89]

    Make llm a testing expert: Bringing human-like interaction to mobile gui testing via functionality-aware decisions

    Zhe Liu, Chunyang Chen, Junjie Wang, Mengzhuo Chen, Boyu Wu, Xing Che, Dandan Wang, and Qing Wang. Make llm a testing expert: Bringing human-like interaction to mobile gui testing via functionality-aware decisions. In Proceedings of the IEEE/ACM 46th International Conference o...

  82. [90]

    Quark: Controllable text generation with reinforced unlearning

    Ximing Lu, Sean Welleck, Jack Hessel, Liwei Jiang, Lianhui Qin, Peter West, Prithviraj Ammanabrolu, and Yejin Choi. Quark: Controllable text generation with reinforced unlearning. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances in Neural...

  83. [91]

    Self-refine: Iterative refinement with self-feedback, 2023

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterativ...

  84. [92]

    Llm-powered conversational voice assistants: Interaction patterns, opportunities, challenges, and design guidelines

    Amama Mahmood, Junxiang Wang, Bingsheng Yao, Dakuo Wang, and Chien-Ming Huang. Llm-powered conversational voice assistants: Interaction patterns, opportunities, challenges, and design guidelines. arXiv preprint arXiv:2309.13879, 2023

  85. [93]

    Mathewson, Jaylen Pittman, and Richard Evans

    Piotr Mirowski, Kory W. Mathewson, Jaylen Pittman, and Richard Evans. Co-writing screenplays and theatre scripts with language models: An evaluation by industry professionals, 2022. URL https://arxiv.org/abs/2209.14958

  86. [94]

    Foundation and large language models: fundamentals, challenges, opportunities, and social impacts

    Devon Myers, Rami Mohawesh, Venkata Ishwarya Chellaboina, Anantha Lakshmi Sathvik, Praveen Venkatesh, Yi-Hui Ho, Hanna Henshaw, Muna Alhawawreh, David Berdik, and Yaser Jararweh. Foundation and large language models: fundamentals, challenges, opportunities, and social impacts....

  87. [95]

    Ollama: Get up and running with Llama 3.1, Mistral, Gemma 2, and other large language models , 2024

    Ollama. Ollama: Get up and running with Llama 3.1, Mistral, Gemma 2, and other large language models , 2024. URL https://github.com/ollama/ollama/tree/main

  88. [96]

    Gpt-4 technical report, 2024

    OpenAI. Gpt-4 technical report, 2024. URL https://arxiv.org/abs/2303.08774

  89. [97]

    Can separators improve chain-of-thought prompting? arXiv, 2024

    Yoonjeong Park, Hyunjin Kim, Chanyeol Choi, Junseong Kim, and Jy-yong Sohn. Can separators improve chain-of-thought prompting? arXiv, 2024. doi:10.48550/arxiv.2402.10645

  90. [98]

    Arkil Patel, Satwik Bhattamishra, and Navin Goyal. Are NLP models really able to solve simple math word problems? In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2080--2094...

  91. [99]

    Refiner: Reasoning feedback on intermediate representations, 2024

    Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, and Boi Faltings. Refiner: Reasoning feedback on intermediate representations, 2024. URL https://arxiv.org/abs/2304.01904

  92. [100]

    Machine translation with large language models: Prompt engineering for persian, english, and russian directions

    Nooshin Pourkamali and Shler Ebrahim Sharifi. Machine translation with large language models: Prompt engineering for persian, english, and russian directions. 2024. URL https://arxiv.org/abs/2401.08429

  93. [101]

    Can github issues be solved with tree of thoughts? arXiv, 2024

    Ricardo La Rosa, Corey Hulse, and Bangdi Liu. Can github issues be solved with tree of thoughts? arXiv, 2024. doi:10.48550/arxiv.2405.13057

  94. [102]

    Can llms master math? investigating large language models on math stack exchange

    Ankit Satpute, Noah Gie ing, Andr\' e Greiner-Petter, Moritz Schubotz, Olaf Teschke, Akiko Aizawa, and Bela Gipp. Can llms master math? investigating large language models on math stack exchange. In Proceedings of the 47th International ACM SIGIR Conference on Research and Dev...

  95. [103]

    Algorithm of thoughts: Enhancing exploration of ideas in large language models

    Bilgehan Sel, Ahmad Al-tawaha, Vanshaj Khattar, Ruoxi Jia, and Ming Jin. Algorithm of thoughts: Enhancing exploration of ideas in large language models. arXiv, 2023. doi:10.48550/arxiv.2308.10379

  96. [104]

    Small llms are weak tool learners: A multi-llm agent

    Weizhou Shen, Chenliang Li, Hongzhan Chen, Ming Yan, Xiaojun Quan, Hehong Chen, Ji Zhang, and Fei Huang. Small llms are weak tool learners: A multi-llm agent. arXiv, 2024. doi:10.48550/arxiv.2401.07324

  97. [105]

    Reflexion: Language agents with verbal reinforcement learning, 2023

    Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning, 2023. URL https://arxiv.org/abs/2303.11366

  98. [106]

    Chain of thoughtlessness? an analysis of cot in planning, 2024

    Kaya Stechly, Karthik Valmeekam, and Subbarao Kambhampati. Chain of thoughtlessness? an analysis of cot in planning, 2024. URL https://arxiv.org/abs/2405.04776

  99. [107]

    Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback

    Niket Tandon, Aman Madaan, Peter Clark, and Yiming Yang. Learning to Repair: Repairing model output errors after deployment using a dynamic memory of feedback . arXiv, 2021. doi:10.48550/arxiv.2112.09737

  100. [108]

    Gemma Team. Gemma. 2024. doi:10.34740/KAGGLE/M/3301. URL https://www.kaggle.com/m/3301

  101. [109]

    Chatgpt as a commenter to the news: can llms generate human-like opinions? In Multidisciplinary International Symposium on Disinformation in Open Online Media, pages 160--174

    Rayden Tseng, Suzan Verberne, and Peter van der Putten. Chatgpt as a commenter to the news: can llms generate human-like opinions? In Multidisciplinary International Symposium on Disinformation in Open Online Media, pages 160--174. Springer, 2023

  102. [110]

    A hybrid intelligence method for argument mining

    Michiel Van Der Meer, Enrico Liscio, Catholijn Jonker, Aske Plaat, Piek Vossen, and Pradeep Murukannaiah. A hybrid intelligence method for argument mining. Journal of Artificial Intelligence Research, 80: 0 1187--1222, 2024

  103. [111]

    A survey of prompt engineering methods in large language models for different nlp tasks, 2024

    Shubham Vatsal and Harsh Dubey. A survey of prompt engineering methods in large language models for different nlp tasks, 2024. URL https://arxiv.org/abs/2407.12994

  104. [112]

    Self-consistency improves chain of thought reasoning in language models

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In ICLR, 2022. URL http://arxiv.org/abs/2203.11171

  105. [113]

    Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents, 2024

    Zihao Wang, Shaofei Cai, Guanzhou Chen, Anji Liu, Xiaojian Ma, and Yitao Liang. Describe, explain, plan and select: Interactive planning with large language models enables open-world multi-task agents, 2024. URL https://arxiv.org/abs/2302.01560

  106. [114]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. arXiv, 2022. doi:10.48550/arxiv.2201.11903

  107. [115]

    Generating sequences by learning to self-correct, 2022

    Sean Welleck, Ximing Lu, Peter West, Faeze Brahman, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. Generating sequences by learning to self-correct, 2022. URL https://arxiv.org/abs/2211.00053

  108. [116]

    Mldt: Multi-level decomposition for complex long-horizon robotic task planning with open-source large language model, 2024

    Yike Wu, Jiatao Zhang, Nan Hu, LanLing Tang, Guilin Qi, Jun Shao, Jie Ren, and Wei Song. Mldt: Multi-level decomposition for complex long-horizon robotic task planning with open-source large language model, 2024. URL https://arxiv.org/abs/2403.18760

  109. [117]

    Self-evaluation guided beam search for reasoning, 2023

    Yuxi Xie, Kenji Kawaguchi, Yiran Zhao, Xu Zhao, Min-Yen Kan, Junxian He, and Qizhe Xie. Self-evaluation guided beam search for reasoning, 2023. URL https://arxiv.org/abs/2305.00633

  110. [118]

    Designing heterogeneous llm agents for financial sentiment analysis

    Frank Xing. Designing heterogeneous llm agents for financial sentiment analysis. ACM Transactions on Management Information Systems, 2024

  111. [119]

    Unlocking the potential: A comprehensive exploration of large language models in natural language processing

    Qing Xue. Unlocking the potential: A comprehensive exploration of large language models in natural language processing. Applied and Computational Engineering, 57 0 (1): 0 247--252, 2024. ISSN 2755-2721. doi:10.54254/2755-2721/57/20241341

  112. [120]

    Generative ai in the era of transformers: Revolutionizing natural language processing with llms

    Archna Balkrishna Yadav. Generative ai in the era of transformers: Revolutionizing natural language processing with llms. Journal of Image Processing and Intelligent Remote Sensing, 0 (42): 0 54--61, 2024. ISSN 2815-0953. doi:10.55529/jipirs.42.54.61. URL https://pdfs.semantic...

  113. [121]

    Qwen2 technical report, 2024

    An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jianxin Yang, Jin Xu, Jingren Zhou, Jinze...

  114. [122]

    Large language models are contrastive reasoners

    Liang Yao. Large language models are contrastive reasoners. arXiv, 2024. doi:10.48550/arxiv.2403.08211

  115. [123]

    Griffiths, Yuan Cao, and Karthik Narasimhan

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36, 2023 a . ISSN 10495258. URL https://arxiv.or...

  116. [124]

    React: Synergizing reasoning and acting in language models, 2023 b

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models, 2023 b . URL https://arxiv.org/abs/2210.03629

  117. [125]

    Leveraging generative ai and large language models: a comprehensive roadmap for healthcare integration

    Ping Yu, Hua Xu, Xia Hu, and Chao Deng. Leveraging generative ai and large language models: a comprehensive roadmap for healthcare integration. In Healthcare, volume 11, page 2776. MDPI, 2023

  118. [126]

    A systematic survey of text summarization: From statistical methods to large language models

    Haopeng Zhang, Philip S Yu, and Jiawei Zhang. A systematic survey of text summarization: From statistical methods to large language models. arXiv preprint arXiv:2406.11289, 2024

  119. [127]

    Exploring collaboration mechanisms for llm agents: A social psychology view

    Jintian Zhang, Xin Xu, and Shumin Deng. Exploring collaboration mechanisms for llm agents: A social psychology view. arXiv preprint arXiv:2310.02124, 2023 a

  120. [128]

    Evaluating the performance of large language models on gaokao benchmark

    Xiaotian Zhang, Chunyang Li, Yi Zong, Zhengyu Ying, Liang He, and Xipeng Qiu. Evaluating the performance of large language models on gaokao benchmark. 2023 b

  121. [129]

    Misinforming llms: vulnerabilities, challenges and opportunities, 2024

    Bo Zhou, Daniel Geißler, and Paul Lukowicz. Misinforming llms: vulnerabilities, challenges and opportunities, 2024. URL https://arxiv.org/abs/2408.01168

  122. [130]

    Least-to-most prompting enables complex reasoning in large language models

    Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and Ed Chi. Least-to-most prompting enables complex reasoning in large language models. The International Conference on Learning Representation...

  123. [131]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION write.doi doi empty "" new.block "DOI: " doi * "" * if FUNCTION write.url doi empty url empty ...

  124. [132]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.sentence output.state after.block = 'skip output.state before.all = 'skip after.sentence '...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.