Pith. sign in

REVIEW 4 major objections 5 minor 6 cited by

CodeAgents: A Token-Efficient Framework for Codified Multi-Agent Reasoning in LLMs

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Codified prompts outscore plain language and slash tokens by 87%

desk verdict A sensible, incremental combination of code-form prompting ideas whose headline accuracy and token-savings claims overshoot what the tables actually show. read the letter →

arxiv 2507.03254 v1 pith:LEQP4T6R submitted 2025-07-04 cs.AI

Bruce Yang , Xinfeng He , Huan Gao , Yifan Cao , Xiaofan Li , David Hsu This is my paper
classification cs.AI
keywords codifiedpromptingmulti-agentreasoningpseudocodeplanningtokenefficiencyfeedback-drivenreplanningassert-basedrecoveryLLMagentsbenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that prompting LLM agents to reason in structured pseudocode, instead of free-form natural language, makes multi-agent planning both more accurate and much cheaper. Across three benchmarks the codified format is reported to beat natural-language prompting by 3–36 percentage points in planning accuracy while cutting input tokens by 55–87% and output tokens by 41–70%. On the VirtualHome household-planning benchmark the method reaches a 56% success rate, described as a new state of the art. A sympathetic reader should care because token cost and context limits are practical bottlenecks for agent deployment, and a format that gains accuracy while spending fewer tokens would be directly useful.

What carries the argument

Codified prompting is the central object: a prompt template written in Python-like pseudocode that the LLM fills in, including typed variables, control structures, comments serving as intermediate thoughts, precondition assertions with else recovery branches, and a structured error-feedback schema. The framework couples this format with two feedback mechanisms—local assert-based recovery for minor execution errors and a global replan loop that consumes a diagnostic trace (failed step, error type, current state) and returns a revised sub-plan. This machinery carries the argument because it increases semantic density, reduces ambiguity, and turns error handling into structured data rather than prose.

What would settle it

Run the codified template and a well-tuned natural-language agent with retrieval, reflection, and replanning on the full GAIA validation set at matched token budgets; if the tuned natural-language agent matches or exceeds the codified approach on accuracy or cost, the claim of consistent superiority is falsified.

Watch

Extended reading notes

Core claim

The paper argues that every component of agent interaction—task descriptions, plans, feedback, system roles, and tool calls—can be encoded as typed, modular pseudocode with loops, conditionals, boolean logic, and embedded assertions, turning a multi-agent dialogue into a reasoning program that one LLM or several specialist roles fill in and follow. In the single-agent setting, assert checks catch local failures and a feedback-driven replan loop regenerates sub-plans after major errors; in the multi-agent setting, Planner, ToolCaller, and Replanner roles exchange codified plans and structured error traces. The reported result is consistent accuracy gains over natural-language baselines—for example, a 10.7-point gain on GAIA with one Gemini model, a 26.8-point gain on the HotpotQA subset in the best configuration, and a 0.56 success rate on VirtualHome—alongside large token and cost reductions on every model tested.

Load-bearing premise

The load-bearing premise is that the paper's self-built natural-language baselines—an untuned default template on GAIA, a reproduced natural-language replanning scheme on VirtualHome, and a 100-example HotpotQA subset—are fair stand-ins for standard practice; if deployed natural-language agents are typically stronger than these baselines, the reported margins could shrink or vanish.

Editorial extensions

If this is right

  • If codified prompts really dominate natural language on these benchmarks, agent prompts should be written as code-like scaffolds rather than open-ended dialogues.
  • Input-token reductions of 55–87% mean agents can complete longer task loops or operate within smaller context windows before hitting limits.
  • Structured error traces make replanning and verification more tractable than free-text reflection, because failures arrive as typed data that can be inspected programmatically.
  • The hybrid result—code with English comments—performs best on VirtualHome, suggesting that light natural-language annotation still contributes to interpretability.
  • Because the gains appear across several model families, the effect is not tied to a single LLM.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's ablations suggest the prompt format itself, not the model, drives most of the gain; a natural next test is whether codified prompts still win against retrieval-augmented, reflection-equipped, or heavily tuned natural-language agents.
  • The reported cost savings imply codified prompting could make frontier models practical for high-frequency agent tasks, a consequence the paper only mentions in passing.
  • The bilingual-comment experiments hint that token density trades against accuracy in a language-dependent way, suggesting an untested extension of choosing comment language to optimize a cost-accuracy frontier.
  • If codified planning traces are machine-checkable, automated verification of agent behavior before execution becomes feasible; the paper gestures toward this direction but does not build the verifier.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes CodeAgents, a prompting framework that represents multi-agent reasoning as structured pseudocode with typed variables, control flow, assertions, and replanning loops. It evaluates the framework on GAIA, HotpotQA, and VirtualHome with several LLMs and reports consistent accuracy gains of 3–36 percentage points, input and output token reductions of 55–87% and 41–70%, and a new state-of-the-art success rate of 56% on VirtualHome. The paper also presents ablations isolating the contributions of code, assertions, replanning, and language comments.

Significance. If the claims were fully supported, the paper would make a useful empirical contribution by highlighting token efficiency as an evaluation dimension for multi-agent prompting and by showing that code-style prompts can reduce cost without sacrificing accuracy. The ablation design is informative, and the explicit measurement of token usage and cost is a strength. However, the evidence as presented supports only a weaker version of the claim: code prompts are generally more token-efficient and sometimes more accurate. Several headline numbers in the abstract are contradicted by the paper's own tables, and the VirtualHome SOTA claim is not statistically grounded against the published baseline.

major comments (4)
  1. [Abstract; Tables 2–4] The abstract's token-efficiency ranges are not supported by the paper's own tables. The abstract claims input and output token reductions of 55–87% and 41–70%, respectively, but Table 2 reports a total-token reduction of 40.7% for VirtualHome and an output-token increase from 217.58 to 484.18; Table 3 reports input reductions of 67.8% and 43.6% and output reductions of 44.5% and 35.2%; and Table 4 reports input reductions of 40.2–72.3% and output reductions of 24.5–61.9%. The claimed ranges should be corrected or replaced by benchmark-specific numbers, and the output-token increase on VirtualHome should be discussed rather than hidden by the total-token figure.
  2. [§3.1, Table 2] The claim of a new state-of-the-art success rate of 56% on VirtualHome is not supported by the presented evidence. Table 2 lists the published BrainBody result as 0.54±0.09 and CodeAgents as 0.56±0.11; these are within one standard deviation of each other, and no significance test or paired comparison is provided. Moreover, the authors' own natural-language reproduction of BrainBody, labeled 'NL(BrainBody)', obtains 0.36±0.05, well below the published 0.54±0.09, which raises the question whether the reproduction follows the original protocol. Without a faithful baseline or a statistical test, the SOTA claim should be withdrawn or re-evidenced.
  3. [§3.3, Table 4] The abstract and Section 3.3 claim consistent improvements, but Table 4 shows Gemini-2.5-Pro achieving identical accuracy (0.51 vs. 0.51) with only a 3.0% relative F1 gain, and the comparison is based on a 100-example HotpotQA subset with no confidence intervals or significance tests. The 'consistent improvements' claim should be restricted to token and cost efficiency plus accuracy improvements on models where they are observed, or supported by statistical testing on a larger sample.
  4. [§3.2, §3.1] The performance gains are measured against baselines that are not representative of deployed natural-language prompting. The GAIA baseline is explicitly described as an untuned, minimal SmolAgents default template 'not designed to reflect state-of-the-art performance' (§3.2), and the VirtualHome natural-language baseline is a self-reproduction whose 0.36 SR is far below the published BrainBody result (§3.1). The 3–36 percentage point gains are therefore relative to weakened comparators; the paper needs to compare against strong natural-language systems or clearly rebrand the contribution as a token-efficiency method rather than an accuracy improvement over standard practice.
minor comments (5)
  1. [Abstract] The abstract contains a typo: 'codifies multi-agent reasoning enables structured' should be 'codifies multi-agent reasoning and enables structured'.
  2. [Table 1] The name 'Codifying-Agent' in Table 1 is inconsistent with the name 'CodeAgents' used throughout the rest of the paper; the terminology should be unified.
  3. [References [11], [21], [22]] The reference entries for the same author use inconsistent name forms ('Shinn Yao' and 'Yao Shinn'); these should be harmonized according to the journal's citation style.
  4. [Figure 4] The code block in Figure 4 contains typographical artifacts such as 'PageUpTool,,→' and misplaced line breaks; these should be cleaned up for readability.
  5. [§4.2] The text 'Addingreplan' is missing a space and should read 'Adding replan'.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: CodeAgents is evaluated on external benchmarks with no fitted parameters, and the central claims are empirical rather than definitional; baseline-construction concerns are validity issues, not circular reasoning.

full rationale

I traced the paper's claimed derivation chain. CodeAgents is a prompting framework; it contains no fitted parameters, no equations that map inputs to outputs by construction, and no author self-citations used as load-bearing support. The central performance claims are measured on external benchmarks (GAIA, HotpotQA, VirtualHome) against either published numbers (ProgPrompt, BrainBody) or self-constructed natural-language baselines. The self-built baselines could be unrepresentative or deliberately weak—the GAIA baseline is described as 'not designed to reflect state-of-the-art performance' and 'untuned and minimal'—but that is a comparison-validity concern, not circularity: the baseline scores are not derived from CodeAgents' own outputs, and no quantity is defined in terms of the quantity it is used to predict. Similarly, the abstract's aggregate token-reduction ranges do not match Tables 2-4 (VirtualHome output tokens increase from 217.58 to 484.18), which is an internal-consistency or reporting problem rather than a circular step. The token savings are measured empirically and are not identity statements; indeed the VirtualHome output-token increase shows that coding does not guarantee lower token counts by definition. The 'new state-of-the-art' VirtualHome claim sits within one standard deviation of the published BrainBody number, but that is statistical over-claiming, not circular self-reference. No step reduces to its own input, no fitted parameter is renamed as a prediction, and no load-bearing premise rests on a self-citation. I therefore find no significant circularity.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The method contributes no fitted numerical parameters and no new physical entities. Its main load-bearing assumptions concern benchmark representativeness, baseline fairness, and the reliability of LLMs at following pseudocode. The hand-written prompt structures, such as assert predicates and loop-break thresholds, act as unquantified design choices rather than fitted constants.

free parameters (2)
  • Replanning loop iteration cap = unspecified
    No maximum number of replanning rounds is stated for GAIA or HotpotQA; the loop may run until success or budget, which affects both accuracy and token counts.
  • Scroll-loop termination condition ('too_many_pages_scrolled') = not defined
    Figure 4 pseudocode breaks the scroll loop on an undefined condition; this hand-chosen threshold is not reported and can affect task success in web-browsing benchmarks.
assumptions (5)
  • domain assumption LLMs can reliably generate and follow structured pseudocode as the prompt templates define.
    The whole method depends on models treating pseudocode as a clearer planning language than prose; Section 2 assumes this without directly measuring code-following reliability.
  • domain assumption The chosen benchmarks and subsets represent the general multi-agent planning settings the paper claims.
    GAIA is restricted to Level 1 validation and HotpotQA to a 100-example subset without a stated seed (Sections 3.2 and 3.3), so generalization across tasks and domains is assumed, not shown.
  • ad hoc to paper The untuned SmolAgents natural-language template is a fair baseline for isolating prompt-format effects.
    Section 3.2 explicitly states the baseline is not designed to reflect state-of-the-art performance, so improvement claims over a deliberately weak baseline are partly by construction.
  • domain assumption Token counts as returned by model APIs are an adequate proxy for cost and efficiency.
    Token tables (Tables 3 and 4) are reported without pricing details or caching effects, and the cost figures assume simple per-token pricing.
  • ad hoc to paper Whitespace, comments, and language choice do not introduce confounds beyond the intended format change.
    The best configuration varies format, comments, and language simultaneously, so attribution of gains to any single mechanism is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of CodeAgents: A Token-Efficient Framework for Codified Multi-Agent Reasoning in LLMs." pith.science (2026). https://pith.science/paper/LEQP4T6R

@misc{pith2026250703254,
  author       = {Pith},
  title        = {Pith review of: CodeAgents: A Token-Efficient Framework for Codified Multi-Agent Reasoning in LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LEQP4T6R}},
  note         = {Machine review of arXiv:2507.03254}
}
read the original abstract

Effective prompt design is essential for improving the planning capabilities of large language model (LLM)-driven agents. However, existing structured prompting strategies are typically limited to single-agent, plan-only settings, and often evaluate performance solely based on task accuracy - overlooking critical factors such as token efficiency, modularity, and scalability in multi-agent environments. To address these limitations, we introduce CodeAgents, a prompting framework that codifies multi-agent reasoning and enables structured, token-efficient planning in multi-agent systems. In CodeAgents, all components of agent interaction - Task, Plan, Feedback, system roles, and external tool invocations - are codified into modular pseudocode enriched with control structures (e.g., loops, conditionals), boolean logic, and typed variables. This design transforms loosely connected agent plans into cohesive, interpretable, and verifiable multi-agent reasoning programs. We evaluate the proposed framework across three diverse benchmarks - GAIA, HotpotQA, and VirtualHome - using a range of representative LLMs. Results show consistent improvements in planning performance, with absolute gains of 3-36 percentage points over natural language prompting baselines. On VirtualHome, our method achieves a new state-of-the-art success rate of 56%. In addition, our approach reduces input and output token usage by 55-87% and 41-70%, respectively, underscoring the importance of token-aware evaluation metrics in the development of scalable multi-agent LLM systems. The code and resources are available at: https://anonymous.4open.science/r/CodifyingAgent-5A86

Figures

Figures reproduced from arXiv: 2507.03254 by the authors.

Figure 1
Figure 1. Architecture of our VirtualHome agent loop. The agent executes modular plans containing [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Example of a fully codified prompt for the task [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. CodeAgents architecture for multi-agent coordination. Agents exchange codified prompts, [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: A simplified codified agent workflow illustrating structured system prompt initialization, [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Accuracy per unit cost on HotpotQA. We evaluate our framework on HotpotQA, a widely used multi-hop QA benchmark requiring multi-document reasoning. A 100-example sub￾set was sampled from the test-fullwiki split to compare models under natural language and codified prom…

Discussion (0). Sign in to comment.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory

    cs.CL 2025-11 unverdicted novelty 7.0 of 10

    Evo-Memory is a new benchmark for self-evolving memory in LLM agents across task streams, with baseline ExpRAG and proposed ReMem method that integrates reasoning, actions, and memory updates for continual improvement.

  2. The Best Programming Language for Tokenmaxxing: An Investigation of Coding Agent Behavior Across Programming Languages

    cs.SE 2026-07 conditional novelty 6.0 of 10

    For five coding agents on 100 competitive-programming tasks, output-token cost varies sharply by language — OCaml costs roughly 1.3–1.7x Python — even after controlling for problem difficulty.

  3. Evo-Memory: Benchmarking LLM Agent Test-time Learning with Self-Evolving Memory

    cs.CL 2025-11 unverdicted novelty 6.0 of 10

    Evo-Memory is a new streaming benchmark and evaluation framework for self-evolving memory in LLM agents, unifying over ten memory modules and introducing the ReMem pipeline for continual improvement on multi-turn and ...

  4. Token Economics for LLM Agents: A Dual-View Study from Computing and Economics

    cs.AI 2026-05 unverdicted novelty 4.0 of 10

    The paper delivers a unified survey of token economics for LLM agents, conceptualizing tokens as production factors, exchange mediums, and units of account across micro, meso, macro, and security dimensions using esta...

  5. Toward Efficient Agents: Memory, Tool learning, and Planning

    cs.AI 2026-01 conditional novelty 3.0 of 10

    A survey that organizes efficiency techniques for LLM agents into memory, tool learning, and planning, and consolidates benchmarks and metrics for measuring cost-performance trade-offs.

  6. Tractable Asymmetric Verification for Large Language Models via Deterministic Replicability

    cs.AI 2025-09 conditional novelty 3.0 of 10

    An LLM output can be verified by regenerating a few randomly chosen segments under identical hardware, with a tunable detection probability and 12.4x speedup over full regeneration.

Reference graph

Works this paper leans on

24 extracted references · 5 canonical work pages · cited by 5 Pith papers

  1. [1]

    Grounding llms for robot task planning using closed-loop state feedback

    Vineet Bhat, Ali Umut Kaypak, Prashanth Krishnamurthy, Ramesh Karri, and Farshad Khorrami. Grounding llms for robot task planning using closed-loop state feedback. arXiv preprint arXiv:2402.08546, 2024. URL https://arxiv.org/abs/2402.08546

    arXiv 2024

  2. [3]

    Metagpt: Meta programming for a multi-agent collaborative framework

    Shixiang Hong, Ming Zhuge, Jie Chen, Xue Zheng, Yuxuan Cheng, Jing Wang, Cheng Zhang, Stephen Yau, Zihan Lin, Lei Zhou, Chunlin Ran, Li Xiao, Chun Wu, and Jürgen Schmidhu- ber. Metagpt: Meta programming for a multi-agent collaborative framework. arXiv preprint arXiv:2308.00352, 2024. URL https://arxiv.org/abs/2308.00352

    arXiv 2024

  3. [4]

    Optimizing Token Consumption in LLMs: A Nano Surge Approach for Code Reasoning Efficiency

    Junwei Hu, Weicheng Zheng, Yan Liu, and Yihan Liu. Token-aware coding flow: A study with nano surge in reasoning model. arXiv preprint arXiv:2504.15989, 2025. URL https: //arxiv.org/abs/2504.15989

    work page Pith review arXiv 2025

  4. [5]

    Inner monologue: Embodied reasoning through planning with language models

    Wenlong Huang, Ishika Singh, Valts Blukis, and Animesh Garg. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05608, 2023. URL https://arxiv.org/abs/2207.05608

    arXiv 2023

  5. [6]

    Camel: Communicative agents for "mind" exploration of large language model society

    Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for "mind" exploration of large language model society. arXiv preprint arXiv:2303.17760, 2023. URL https://arxiv.org/abs/2303.17760

    arXiv 2023

  6. [8]

    Gaia: A benchmark for general ai assistants

    Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, and Thomas Scialom. Gaia: A benchmark for general ai assistants. arXiv preprint arXiv:2311.12983, 2023. URL https://arxiv.org/abs/2311.12983

    arXiv 2023

  7. [9]

    Virtualhome: Simulating household activities via programs

    Xavier Puig, Kevin Ra, Marko Boben, Jiaman Li, Tingwu Wang, Sanja Fidler, and Antonio Torralba. Virtualhome: Simulating household activities via programs. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) , pages 8494– 8502, 2018. URL https://openaccess.thecvf.com/content_cvpr_2018/html/Puig_ VirtualHome_Simulating_Ho...

    work page 2018

  8. [10]

    smolagents: a smol library to build great agentic systems

    Aymeric Roucher, Albert Villanova del Moral, Thomas Wolf, Leandro von Werra, and Erik Kaunismäki. smolagents: a smol library to build great agentic systems. https://github. com/huggingface/smolagents, 2024. Accessed: 2025-05-11. 11

    work page 2024

Show all 24 references
  1. [12]

    Progprompt: Generating situated robot task plans using large language models

    Ishika Singh, Valts Blukis, Arsalan Mousavian, Ayush Goyal, Danfei Xu, and Jonathan Tremblay. Progprompt: Generating situated robot task plans using large language models. arXiv preprint arXiv:2209.11302, 2023. URL https://arxiv.org/abs/2209.11302

    2023 arXiv

  2. [13]

    Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models

    Liang Wang, Wenxuan Xu, Yanyan Lan, Zhiyuan Hu, Yiming Lan, Raymond Lee, and Ee-Peng Lim. Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. arXiv preprint arXiv:2305.04091, 2023. URL https://arxiv.org/abs/ 2305.04091

    2023 arXiv

  3. [14]

    URL https://arxiv.org/abs/2303.11366

    arXiv

  4. [15]

    Executable code actions elicit better llm agents

    Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. Executable code actions elicit better llm agents. In Proceedings of the 41st International Conference on Machine Learning (ICML) , 2024. URL https://arxiv.org/abs/2402. 01030

    2024

  5. [16]

    Talk structurally, act hierarchically: A collaborative framework for llm multi-agent systems

    Zhao Wang, Sota Moriyama, Wei-Yao Wang, Briti Gangopadhyay, and Shingo Takamatsu. Talk structurally, act hierarchically: A collaborative framework for llm multi-agent systems. arXiv preprint arXiv:2502.11098, 2025. URL https://arxiv.org/abs/2502.11098

    2025 arXiv

  6. [17]

    Megaagent: A practical framework for autonomous cooperation in large-scale llm agent systems.arXiv preprint arXiv:2408.09955, 2024

    Qian Wang, Tianyu Wang, Qinbin Li, Jingsheng Liang, and Bingsheng He. Megaagent: A practical framework for autonomous cooperation in large-scale llm agent systems.arXiv preprint arXiv:2408.09955, 2024. URL https://arxiv.org/abs/2408.09955

    2024 arXiv

  7. [18]

    Codeplan: Unlocking reasoning potential in large language models by scaling code-form planning

    Jiarui Wen, Jiahua Guan, Haoyu Wang, Wenhao Wu, and Minlie Huang. Codeplan: Unlocking reasoning potential in large language models by scaling code-form planning. arXiv preprint arXiv:2409.12452, 2025. URL https://arxiv.org/abs/2409.12452

    2025 arXiv

  8. [19]

    Autogen: Enabling next-gen llm applications via multi-agent conversation

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al. Autogen: Enabling next-gen llm applications via multi-agent conversation. arXiv preprint arXiv:2308.08155, 2023. URL https://arxiv. org/abs/2308.08155

    2023 arXiv

  9. [20]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903 , 2022. URL https://arxiv.org/abs/2201. 11903

    2022 arXiv

  10. [21]

    React: Syner- gizing reasoning and acting in language models

    Shinn Yao, Jeffrey Zhao, Dian Yu, Yanfei Zhao, Yi Zhang, and Dongyan Zhao. React: Syner- gizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022. URL https://arxiv.org/abs/2210.03629

    2022 arXiv

  11. [23]

    Cohen, Ruslan Salakhut- dinov, and Christopher D

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhut- dinov, and Christopher D. Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language ...

    2018

  12. [24]

    Solving challenging math word problems using gpt-4 code interpreter with code-based self-verification

    Aojun Zhou, Kai Wang, Zhi Lu, Weijia Shi, Shuhan Luo, Zhaohui Qin, Shuchang Lu, An Jia, Yifan Liu, Yichong Zhang, et al. Solving challenging math word problems using gpt-4 code interpreter with code-based self-verification. arXiv preprint arXiv:2308.07921, 2023. URL https://ar...

    2023 arXiv

  13. [26]

    URL https://arxiv.org/abs/2305.10601

    arXiv

  14. [27]

    Algorithm of thoughts: En- hancing exploration of ideas in large language models

    Zhuosheng Zhang, Aston Zhang, Mu Li, and Alexander Smola. Algorithm of thoughts: En- hancing exploration of ideas in large language models. arXiv preprint arXiv:2308.10379, 2023. URL https://arxiv.org/abs/2308.10379

    2023 arXiv

  15. [2023]

    URL https://arxiv.org/abs/2211.10435

    arXiv

  16. [2024]

    URL https://arxiv.org/abs/2404.02575

    arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.