Pith. sign in

REVIEW 4 major objections 6 minor 31 references

Decompose, Plan in Parallel, and Merge: A Novel Paradigm for Large Language Models based Planning with Multiple Constraints

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read By splitting a trip into four aspect-specific subtasks, planning those in parallel, and merging with verification, DPPM claims far higher all-constraint satisfaction than prior LLM planners.

desk verdict DPPM is a clean combination of known building blocks that shows large, consistent gains on TravelPlanner, but the unreleased ChinaTravel-M adaptation is too shaky to count as evidence. read the letter →

arxiv 2506.02683 v1 pith:5BP6GWJT submitted 2025-06-03 cs.CL

Zhengdong Lu , Weikai Lu , Yiling Tao , Yun Dai , ZiXuan Chen , Huiping Zhuang , Cen Chen , Hao Peng
show 1 more author
Ziqian Zeng
This is my paper
classification cs.CL
keywords LLMplanningtravelconstraintsatisfactiontaskdecompositionparallelplanmergingverificationandrefinementbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that LLM-based planning fails under many constraints because existing methods either pile all constraints onto one step or let one subtask's mistakes contaminate the next. It proposes DPPM, a decomposition-plan-merge pipeline for travel planning: split the trip into four subtasks (transportation, accommodation, attractions, meals), have a separate local agent plan each subtask in parallel with only its own constraints, then merge the subplans in a fixed order with a verification-and-refinement loop that catches violations. On TravelPlanner, DPPM's final pass rate is 58.9% with Qwen2.5-32B versus 36.7% for LLM-Modulo, and on ChinaTravel-M it beats LLM-Modulo by 12.8 to 32.0 points across difficulty levels. The claim matters because it suggests that decomposing constraints by aspect, rather than by time, removes the cascade and overload that plague sequential and interleaved planners.

What carries the argument

The load-bearing mechanism is the constraint-aware decomposition of the overall trip into four aspect subtasks. Each subtask is handled by its own local agent with a custom prompt, produces multiple candidate subplans under high-temperature sampling, and is checked by a per-aspect critic function. The incremental merge then combines transportation with attractions first, adds accommodation, and finishes with meals, cycling through the Cartesian product of candidate subplans until a combination passes all checks. A verification and refinement module feeds critic output back to the generating agent for up to ten re-planning iterations, and the Global Constraint Instruction pushes each agent to keep budget and stay flexible enough for the other aspects.

What would settle it

Take a set of planning queries whose constraints are deliberately cross-aspect, for example 'the hotel must be within walking distance of the lunch restaurant, and the total daily walking distance under the attraction plan is capped', and count the final pass rate. If DPPM's advantage over a sequential planner with the same refinement budget disappears on such queries, the aspect-separability assumption is the failing premise.

Watch

Extended reading notes

Core claim

The central discovery is that a travel plan's constraints can be assigned to four independent aspects and planned in parallel. Each local agent receives only its aspect's constraints plus a global constraint instruction to leave room for other aspects; this avoids the heavy-constraint bottleneck. The subsequent incremental merge proceeds from transportation-plus-attractions to accommodation to meals, using the Cartesian product of candidate subplans to find a consistent combination, and a verification/refinement module checks hard and commonsense constraints at both local and merged stages. Reported results: final pass rate rises to 58.9%, 64.4%, 76.7%, and 87.2% on TravelPlanner across Qwen2.5-32B, Qwen2.5-72B, DeepSeek-V3, and DeepSeek-R1, versus 36.7%, 39.4%, and 57.8% for LLM-Modulo where it was run; ChinaTravel-M with Qwen2.5-32B reaches 87.1%, 70.7%, and 52.5% on easy, medium, and human difficulties, versus 55.8%, 38.7%, and 39.7% for LLM-Modulo.

Load-bearing premise

The load-bearing premise is that a travel plan's constraints split cleanly into four independent aspects, transportation, accommodation, attractions, and meals, so every cross-aspect conflict can be absorbed by the incremental merge and refinement; for queries where one constraint inherently belongs to two aspects, the subplans may be mutually incompatible from the start.

Editorial extensions

If this is right

  • On TravelPlanner, DPPM reaches 58.9% final pass rate with Qwen2.5-32B and 76.7% with DeepSeek-V3, well above Direct, CoT, and LLM-Modulo baselines.
  • On ChinaTravel-M with Qwen2.5-32B, DPPM surpasses LLM-Modulo by 31.3 points on easy, 32.0 on medium, and 12.8 on human-level samples.
  • The verification and refinement module is the largest single contributor: removing it cuts final pass rate from 58.9% to 20.0% on TravelPlanner; removing the Cartesian product cuts it to 46.1% and removing the global-constraint instruction to 49.4%.
  • DPPM degrades more slowly as constraints accumulate: on hard 12-constraint samples it satisfies all constraints in twice as many plans as LLM-Modulo.
  • The authors state the method is tailored to multi-constraint tasks; for single-constraint planning it has no advantage, so the applicability is deliberately bounded.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An implication the authors leave implicit is that the same aspect-based decomposition should transfer to other constrained scheduling domains whose constraints factor into a few independent dimensions, such as event scheduling, logistics routing, or itinerary timetabling, provided cross-dimension interactions remain sparse; the paper only demonstrates travel.
  • Because DPPM gains most on macro pass rates, the harder failure mode for LLMs may be satisfying many constraints at once rather than reasoning about any one; a testable prediction is that the gap over sequential baselines widens as the number of constraints per sample grows.
  • The method is effectively a small implicit search over the Cartesian product of candidate subplans; a cheap extension would be to score candidate combinations by estimated cost before invoking the merge LLM, reducing the number of merge calls.
  • The ablation attributes most of the gain to verification and refinement, so a direct comparison that gives LLM-Modulo the same number of refinement iterations and candidate diversity would isolate how much the parallel decomposition itself contributes.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes DPPM, a three-stage planning paradigm for LLM-based travel planning: constraint-aware task decomposition (grouping all constraints into transportation, accommodation, attraction, and meals), parallel generation of multiple candidate subplans by local agents, and incremental merging of subplans via a Cartesian product with verification and refinement. The method is evaluated on the TravelPlanner validation set and on an adapted, unreleased version of ChinaTravel called ChinaTravel-M, using Qwen2.5-32B, Qwen2.5-72B, DeepSeek-V3, and DeepSeek-R1. The central empirical claim is that DPPM substantially outperforms Direct, CoT, and LLM-Modulo in final pass rate, with reported gains of about 22 to 32 percentage points over LLM-Modulo on TravelPlanner and 13 to 32 points on ChinaTravel-M. The paper also reports ablations showing the contribution of the global constraint instruction, the verification-and-refinement module, and the Cartesian product combination.

Significance. The proposed paradigm is a sensible response to two real limitations of LLM planning: constraint overload and cascading errors. If the reported results are reproducible, the work would be a useful contribution to the travel-planning benchmark literature, with a simple and generalizable decomposition idea. The paper has several strengths: it evaluates across multiple backbone families; it compares against LLM-Modulo using the same constraint evaluation functions and verification loop, making that comparison fair; the ablations isolate the contribution of each component; and the consistency of the TravelPlanner results across backbones is encouraging. However, the current evidence base is not yet sufficient to fully support the abstract's broad claim of significant superiority. The ChinaTravel-M benchmark is model-constructed, undocumented, and unreleased, and the quantitative claims are based on single runs on validation sets without significance tests or error bars. The significance of the work is therefore conditional on the authors providing reproducible benchmark artifacts and statistical grounding.

major comments (4)
  1. [Section 4.1 / Appendix A] The ChinaTravel-M adaptation is not reproducible and may be favorable to DPPM by construction. Appendix A describes only three high-level steps: semantic parsing, iterative refinement, and standardized mapping, with no prompts, no model, no code, no equivalence checks between the original Constraint Expressions and the resulting TravelPlanner-style constraints, and no release of the converted benchmark. Since DPPM's decomposition stage in Section 3.1 assumes every constraint belongs to exactly one of four aspects, a reformatting that reshapes the original CEs into this schema would make the comparison favorable by construction. The authors should release the exact transformation artifacts, report conversion statistics (how many constraints were kept, split, merged, or dropped), verify that no cross-aspect coupling is introduced or removed, and ideally also evaluate on the original ChinaTravel benchmark with its original evaluation process.
  2. [Table 1 / Section 4.2] The central claim of 'significantly outperforms' is not supported by statistical evidence. Table 1 reports point estimates on the TravelPlanner validation set only, with no error bars, no multiple seeds, and no significance tests. The same is true for Table 2 and Figure 5. Given that the headline differences are large, the claim may survive such testing, but the manuscript should either include standard deviations and statistical tests across at least three independent runs, or weaken the wording from 'significantly' to 'consistently outperforms on the validation set'. The authors should also report results on the TravelPlanner test set or justify why validation-only results are sufficient.
  3. [Section 3.1 / Section 3.3] The four-aspect decomposition and the fixed merge order assume that all constraints can be cleanly assigned to one aspect, with only budget and minimum-stay treated as global. The paper does not analyze what happens when constraints couple aspects in ways beyond these two examples, such as 'accommodation must be near a chosen attraction' or 'meals should be in the same city as the day's activities.' Because the ChinaTravel-M adaptation is undocumented, it is unclear whether such cross-aspect constraints existed in the original ChinaTravel and were removed or altered during conversion. Without evidence on non-decomposable or strongly coupled constraints, the paradigm's claimed generality over 'multiple constraints' is not established. The authors should either provide a formal or empirical analysis of the decomposition assumption on the original ChinaTravel constraints, or explicitly scope the method to constraints that satisfy the four-aspect decomposition.
  4. [Section 1 / Section 4.1] The introduction motivates the work by stating that ReAct and Reflexion achieve a final pass rate below 1% on multi-constraint travel planning, but neither baseline appears in Tables 1 or 2. Since the abstract claims DPPM 'significantly outperforms existing methods,' the comparison set is incomplete. The authors should either include these baselines under the same experimental settings or qualify the claim to 'outperforms the compared baselines' throughout the abstract and conclusion.
minor comments (6)
  1. [Table 2] The table title says 'ChinaTravel dataset' but the body and Section 4.1 refer to the adapted 'ChinaTravel-M' dataset; the title should be corrected to avoid ambiguity.
  2. [Introduction / Table 2] The introduction reports improvements on 'hard-level samples,' but Table 2 labels the difficulty levels as Easy, Medium, and Human. The terminology should be aligned, and the 'Human' level should be described more clearly.
  3. [Appendix C] The constraint definitions appendix says the definitions are 'shown in Figure 4,' but the corresponding figure appears to be Figure 7, and the actual constraint definitions are not reproduced in the text. The cross-reference should be fixed and the definitions should be included for self-containedness.
  4. [Section 4.2] The authors remove phone numbers, website links, and geocoordinates from the reference information for all methods. Since this changes the inputs to all methods, the paper should document whether any constraint evaluation functions depend on the removed fields and should report whether results on the unmodified reference information differ.
  5. [Throughout] There are several typographical and grammatical issues, including 'TavelPlanner' in Section 4.1, 'we delegates different types of subtasks' in Section 3.2, and 'an significant step forward' in the Limitations section. These should be corrected.
  6. [References] References Gundawar et al. 2024a and 2024b are identical (arXiv:2405.20625); they should be unified into a single reference or the two distinct papers should be cited separately.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the TravelPlanner evaluation is external and the shared generate-test loop is disclosed; the ChinaTravel-M adaptation is an evaluation caveat, not a derivation-level circularity.

full rationale

DPPM's central claim is supported by scores on the TravelPlanner validation set computed with the benchmark's own fixed constraint evaluators; the paper fits no parameter to these scores and defines no equation in terms of them. The use of the same evaluators for refinement and final scoring (Section 4.2: 'our method employs the same constraint evaluation functions and verification result formats as LLM-Modulo') is a generate-test protocol shared with the baseline, so the comparison is fair rather than circular. I separately weighed the ChinaTravel-M adaptation in Appendix A: because it is produced by model-guided semantic parsing and iterative refinement with no released code or equivalence check, the 12.8-32.0 point ChinaTravel claims are a reproducibility and validity risk. However, that is not an exhibited circular reduction: the paper does not show an equation or fitted quantity that makes the adapted benchmark equivalent to DPPM's output, and the TravelPlanner results are self-contained against an external benchmark. No load-bearing self-citation or imported uniqueness theorem appears. Therefore no circularity step is identified and the score is 0.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical or mathematical entities. Its design choices (aspect grouping, iteration limit, number of subplans) are hand-set but not fitted to the evaluation data. The main assumptions are about the completeness of the benchmark's constraint evaluators and the sufficiency of the four-aspect decomposition.

free parameters (2)
  • Maximum refinement iterations = 10
    Set by hand; Figure 5 shows performance saturates with iterations, so the choice affects the reported ceiling.
  • Number of candidate subplans per local agent = at least 2 (not disclosed)
    Prompts ask for multiple plans; the Cartesian product size grows with this number, so it affects merge cost and final pass rate.
assumptions (3)
  • domain assumption Constraint evaluation functions are correct and complete.
    Used both for refinement feedback and final scoring (Sections 3.4, 4.1); any incompleteness inflates pass rates.
  • domain assumption The four aspects (transportation, accommodation, attractions, meals) cover all relevant constraints in the benchmarks.
    Section 3.1 decomposes all constraints into these groups; constraints that span aspects are treated as global and handled only in merge.
  • domain assumption A local LLM can satisfy a reduced constraint subset reliably.
    The whole benefit of decomposition depends on local agents succeeding with fewer constraints; if they still fail, DPPM would not improve over monolithic planning.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Decompose, Plan in Parallel, and Merge: A Novel Paradigm for Large Language Models based Planning with Multiple Constraints." pith.science (2026). https://pith.science/paper/5BP6GWJT

@misc{pith2026250602683,
  author       = {Pith},
  title        = {Pith review of: Decompose, Plan in Parallel, and Merge: A Novel Paradigm for Large Language Models based Planning with Multiple Constraints},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/5BP6GWJT}},
  note         = {Machine review of arXiv:2506.02683}
}
read the original abstract

Despite significant advances in Large Language Models (LLMs), planning tasks still present challenges for LLM-based agents. Existing planning methods face two key limitations: heavy constraints and cascading errors. To address these limitations, we propose a novel parallel planning paradigm, which Decomposes, Plans for subtasks in Parallel, and Merges subplans into a final plan (DPPM). Specifically, DPPM decomposes the complex task based on constraints into subtasks, generates the subplan for each subtask in parallel, and merges them into a global plan. In addition, our approach incorporates a verification and refinement module, enabling error correction and conflict resolution. Experimental results demonstrate that DPPM significantly outperforms existing methods in travel planning tasks.

Figures

Figures reproduced from arXiv: 2506.02683 by the authors.

Figure 1
Figure 1. Comparison of different types of plan￾ning framework, including sequential decomposition￾planning, interleaved decomposition-planning, and DPPM. Taking travel planning as an example, sequen￾tial methods might first decompose a complex task into temporal segments (e.g., daily itineraries) and then gen￾erate plans for each day in sequence, with each day’s plan building upon previous decisions. Interleaved decompositio… view at source ↗
Figure 2
Figure 2. An overview of our proposed DPPM, which consists of three key stages. (1) In the Constraint-based [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. An overview of the Verification and Refine [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The number of passed samples across the number of constraints for baselines and our method on Hard-level samples of TravelPlanner [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Comparison of Final Pass Rate between baselines and our method across the number of self￾refinement iterations on TravelPlanner dataset. The base model is Qwen2.5-32B-Instruct. 4.3 Main Results In [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Comparison of Pass Rate for each constraint [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Constraints Definition. D Number of Parameters in Base Models The number of parameters in each base model is shown in [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

31 extracted references · 3 canonical work pages

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  4. [4]

    Ron Alterovitz, Sven Koenig, and Maxim Likhachev. 2016. Robot planning in the real world: Research challenges and opportunities. Ai Magazine, 37(2):76--84

    work page 2016

  5. [5]

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and 1 others. 2024. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17682--17690

    2024

  6. [6]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, and 1 others. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  7. [7]

    S \'e bastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, and 1 others. 2024. Sparks of artificial general intelligence: Early experiments with gpt-4. arxiv 2023. arXiv preprint arXiv:2303.12712, 10

    arXiv 2024

  8. [8]

    Wenhu Chen, Xueguang Ma, Xinyi Wang, and William W Cohen. 2022. Program of thoughts prompting: Disentangling computation from reasoning for numerical reasoning tasks. arXiv preprint arXiv:2211.12588

    arXiv 2022

Show all 31 references
  1. [9]

    Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, and 1 others. 2022. A survey on in-context learning. arXiv preprint arXiv:2301.00234

    2022 arXiv

  2. [11]

    Atharva Gundawar, Mudit Verma, Lin Guan, Karthik Valmeekam, Siddhant Bhambri, and Subbarao Kambhampati. 2024 b . Robust planning with llm-modulo framework: Case study in travel planning. arXiv preprint arXiv:2405.20625

    2024 arXiv

  3. [12]

    Taicheng Guo, Kehan Guo, Zhengwen Liang, Zhichun Guo, Nitesh V Chawla, Olaf Wiest, Xiangliang Zhang, and 1 others. 2023. What indeed can gpt models do in chemistry? a comprehensive benchmark on eight tasks. arXiv preprint arXiv:2305.18365, 3

    2023 arXiv

  4. [13]

    Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, and 1 others. 2023. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352, 3(4):6

    2023 arXiv

  5. [14]

    Xu Huang, Weiwen Liu, Xiaolong Chen, Xingmei Wang, Hao Wang, Defu Lian, Yasheng Wang, Ruiming Tang, and Enhong Chen. 2024. Understanding the planning of llm agents: A survey. arXiv preprint arXiv:2402.02716

    2024 arXiv

  6. [15]

    Subbarao Kambhampati, Karthik Valmeekam, Lin Guan, Mudit Verma, Kaya Stechly, Siddhant Bhambri, Lucas Paul Saldyt, and Anil B Murthy. 2024. Position: LLM s can’t plan, but can help planning in LLM -modulo frameworks. In Proceedings of the 41st International Conference on Machi...

    2024

  7. [16]

    Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, and Jianfeng Gao. 2023. Chameleon: Plug-and-play compositional reasoning with large language models. Advances in Neural Information Processing Systems, 36:43447--43478

    2023

  8. [17]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, and 1 others. 2023. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36:46534--46594

    2023

  9. [18]

    Joon Sung Park, Joseph O'Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2023. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th annual acm symposium on user interface software and technology, pages 1--22

    2023

  10. [19]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dess \` , Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. Advances in Neural Information Processing Systems, 36:68539--68551

    2023

  11. [20]

    Jie-Jing Shao, Xiao-Wen Yang, Bo-Wen Zhang, Baizhi Chen, Wen-Da Wei, Guohao Cai, Zhenhua Dong, Lan-Zhe Guo, and Yu-feng Li. 2024. Chinatravel: A real-world benchmark for language agents in chinese travel planning. arXiv preprint arXiv:2412.13682

    2024 arXiv

  12. [21]

    Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023. Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face. Advances in Neural Information Processing Systems, 36:38154--38180

    2023

  13. [22]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634--8652

    2023

  14. [23]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, and 1 others. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  15. [24]

    Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim. 2023. Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. arXiv preprint arXiv:2305.04091

    2023 arXiv

  16. [25]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, and 1 others. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  17. [26]

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, and 1 others. 2025. The rise and potential of large language model based agents: A survey. Science China Information Sciences, 68(2):121101

    2025

  18. [27]

    Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. 2024. Travelplanner: A benchmark for real-world planning with language agents. arXiv preprint arXiv:2402.01622

    2024 arXiv

  19. [28]

    Zelai Xu, Chao Yu, Fei Fang, Yu Wang, and Yi Wu. 2023. Language agents with reinforcement learning for strategic play in the werewolf game. arXiv preprint arXiv:2310.18940

    2023 arXiv

  20. [29]

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388

    2025 arXiv

  21. [30]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2023 a . Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 36:11809--11822

    2023

  22. [31]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023 b . React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR)

    2023

  23. [32]

    Denny Zhou, Nathanael Sch \"a rli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Claire Cui, Olivier Bousquet, Quoc Le, and 1 others. 2022. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625

    2022 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.