Pith. sign in

REVIEW 3 major objections 5 minor 2 cited by

InstructRAG: Leveraging Retrieval-Augmented Generation on Instruction Graphs for LLM-Based Task Planning

T0 review · 3 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read InstructRAG claims that storing past planning actions as a graph of reusable instruction nodes and learning to recombine them with reinforcement and meta-learning improves LLM task planning by up to 19.2% over the best baseline.

desk verdict Instruction graph is a genuine new mechanism, but the headline improvement is overstated and hyperparameters are tuned on the test set. read the letter →

arxiv 2504.13032 v1 pith:KARGGWTW submitted 2025-04-17 cs.AI cs.IR

Zheng Wang , Shu Xian Teo , Jun Jie Chew , Wei Shi This is my paper · ORCID
classification cs.AIcs.IR
keywords largelanguagemodelretrieval-augmentedgenerationagentplanninginstructiongraphmeta-reinforcementlearningenlargabilitytransferabilityfew-shot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that retrieval-augmented generation for LLM task planning should retrieve at the level of individual instruction steps rather than whole trajectories. It identifies two properties a planning RAG needs: enlargability, the ability to answer new questions by recombining previously stored instructions, and transferability, the ability to adapt quickly to entirely new tasks. To get both, InstructRAG builds an instruction graph from successful past plans, trains an RL-Agent to traverse the graph and propose candidate instruction paths, and trains an ML-Agent to meta-learn which candidate path will best help the LLM answer. The two agents are trained end-to-end so that the LLM's final answer score rewards path selection. If the central claim is right, a planning agent can do better by reorganizing its memory as reusable action components than by retrieving full past experiences, with reported gains as large as 19.2% over the best comparison method.

What carries the argument

The load-bearing object is the instruction graph, a directed graph in which each node is a set of similar instructions (merged by an AKNN similarity check against a threshold δ) and each edge is a set of tasks and questions connected by those instructions. Adjacent instructions are never placed in the same node, and similar instructions from different tasks can merge into one node, forming junction nodes where new paths can be spliced together. On top of the graph, an RL-Agent conducts depth-first traversal, deciding include or exclude at each node from a state of three cosine similarities between the question and the node's instructions, edge tasks, and exemplar questions, and receives the LLM's end-to-end metric, such as F1, as reward. An ML-Agent then ranks the K candidate paths by learned question-path similarity, builds the prompt, and is adapted to new tasks with MAML-style meta-updates. The graph provides compositionality, while the two agents provide learned path search and learned prompt selection.

What would settle it

On a HotpotQA test split where no question shares any entity with any instruction stored in the graph, if InstructRAG's F1 drops to the level of whole-trajectory retrieval, such as RAP, then the graph's recombination is not what produces new valid plans, and the enlargability claim would be falsified.

Watch

Extended reading notes

Core claim

On the author's own terms, the discovery is that a RAG planner is materially improved when its external database is an instruction graph—nodes cluster similar atomic instructions such as Search[entity] and Lookup[keyword], edges record the tasks those instructions were part of—because the graph turns stored plans into composable parts. The RL-Agent treats traversal as an MDP and learns whether to include or exclude each node, retrieving K candidate paths; the ML-Agent uses contrastive question-path alignment plus meta-learning to pick the single path as an in-context exemplar. Together they explicitly optimize the two properties the paper names: enlargability (new questions answered by combining stored instructions) and transferability (few-shot adaptation to unseen tasks). Across HotpotQA, ALFWorld, Webshop, and ScienceWorld, and over three backbone LLMs, the system reports consistent gains over RAP, the strongest comparison, including 19.2% on HotpotQA with DeepSeek-V2.

Load-bearing premise

The load-bearing premise is that individual instructions like Search[entity], Lookup[keyword], and GoTo[room] are reusable building blocks, so splicing stored instructions into a new sequence yields a valid plan even when the exact combination of entities in the new question was never seen together in any stored path.

Editorial extensions

If this is right

  • Organizing past experience as a graph of composable instruction steps, instead of retrieving whole trajectories, is what drives the reported gains: removing the graph costs 11.6% on HotpotQA, removing the RL-Agent costs 11.1%, and removing the ML-Agent costs 6.9%.
  • The improvements are not limited to the training distribution: InstructRAG beats RAP on unseen HotpotQA, ALFWorld, and Webshop tasks, and a HotpotQA-trained model generalizes to ScienceWorld with 6–10% gains.
  • The same relative gains appear with three different backbone LLMs, so the mechanism is not tied to one model's reasoning style.
  • A RAG planner built this way tolerates polluted memory: injecting failed instruction paths up to a 50% noise rate degrades F1 by only 11.1%.
  • Few-shot adaptation to new tasks is fast in practice: per-task adaptation takes about 27.1 minutes and performance plateaus once about 80% of support samples are used.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the junction-node mechanism treats the action vocabulary as a small set of reusable operators over a large entity space; if that is the real source of gain, the same graph recombination should transfer to other plan domains with compositional action vocabularies, such as tool-use coding or robot manipulation.
  • Editorial inference: because the RL reward is the answer score of one specific LLM, the learned path policy is coupled to that LLM; porting the trained agents to a new backbone would likely require re-running the reward loop rather than reusing precomputed paths.
  • Editorial inference: the threshold δ controls how aggressively instructions merge into junction nodes, and the paper fixes it at 0.4; an adaptive per-node threshold that reflects how often a node is reused across tasks is a direct, testable extension that could either sharpen or bound the enlargability gain.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. This paper addresses retrieval-augmented generation (RAG) for LLM-based task planning and identifies two challenges: enlargability (recombining stored instruction paths to cover new questions within a task family) and transferability (rapid adaptation to unseen tasks). The proposed InstructRAG builds a directed instruction graph from correct instruction paths extracted on a support set, trains an RL-Agent (REINFORCE with a warm-start phase) to traverse the graph and retrieve K candidate paths, and trains an ML-Agent (contrastive pre-training plus hard-negative fine-tuning) to select one path for use as an in-context exemplar in the prompt to the planning LLM; the two agents are optimized jointly in a MAML-style outer loop. Experiments on HotpotQA, ALFWorld, Webshop, and ScienceWorld with GLM-4, GPT-4o mini, and DeepSeek-V2 report consistent gains over ReAct, WKM, Reflexion, GenGround, and RAP, together with ablations, a noise-robustness study, parameter studies for the clustering threshold delta and the candidate-path count K, and few-shot adaptation curves.

Significance. If the empirical claims hold, the paper makes a useful contribution: it formalizes instruction-level reuse in RAG planning, and its component-wise ablations (Table 5), robustness to noisy historical paths (Table 4), cross-dataset transfer evidence (Table 2), and detailed qualitative trajectory comparisons (Tables 12-14) are well-conceived and largely support the internal logic of the design. The complete listings of Algorithms 1-4 and the explicit meta-reinforcement-learning training recipe make the method concretely reproducible in principle, and the few-shot adaptation experiments (Figure 2) directly probe the transferability property that the paper defines. The significance is tempered, however, by the way the headline improvement is computed and by hyperparameter selection on the reported test metric; the current evidence supports 'competitive with, and often better than, the evaluated baselines', but not yet the abstract's 'best existing approach' claim.

major comments (3)
  1. [Abstract; §5.2(1), Table 1; §2] The headline claim of 'up to a 19.2% improvement over the best existing approach' is not supported by Table 1 as reported. The 19.2% HotpotQA figure is computed against RAP, but RAP is not the best baseline in two of the three HotpotQA rows: with GPT-4o mini, GenGround achieves 28.99 versus RAP's 27.66, and with DeepSeek-V2, GenGround achieves 33.50 versus RAP's 29.82. Against the per-cell best baseline, the average HotpotQA improvement is roughly 7-11%, not 19.2%, so the phrase in §5.2(1), 'over the best baseline (RAP)', is internally inconsistent with the table. In addition, KnowAgent [42] and RAT [33], both discussed in Section 2 as directly relevant TAO-style planning methods (with RAT being RAG-based), are not evaluated, and the GenGround baseline is implemented as a LlamaIndex retriever over stored TAO triplets rather than its published generate-then-ground procedure. The authors should add the missing baselines or explicitly rescope the claim to the evaluated baseline set.
  2. [§5.1 Implementation Details; §5.2(6)-(7), Tables 6-7] The hyperparameters delta=0.4 and K=3 are selected from Tables 6-7, which report the same HotpotQA F1 metric that constitutes the paper's headline result, and no validation split or hyperparameter-selection protocol is described in the experimental setup. This means the reported headline numbers partly reflect test-set fitting. The authors should either select delta and K on a held-out validation split and report the corresponding test numbers, or show that the advantage over baselines is robust to alternative settings; notably, Table 7 already shows that even K=1 (F1 34.78 on DeepSeek-V2) beats GenGround (33.50), so a validation-based selection is likely to preserve the main conclusion, but the current reporting convention is methodologically problematic.
  3. [§5.1 Evaluation Metrics; §5.2(5), Table 5] The statement that 'all reported experimental results are statistically significant, verified by a t-test with p < 0.05' cannot be checked from the paper: the tables report a single F1, success-rate, or reward number per method and backbone, with no indication of the number of runs, the variance, or whether the test was paired per task or per question. Please specify the test design and report the underlying dispersion (for example, per-task scores or standard deviations across runs). In addition, the ablation attribution in Section 5.2(5) uses inconsistent denominators: the 11.6% attributed to removing the instruction graph is relative to the full model, while the 11.1% and 6.9% attributed to removing the RL-Agent and ML-Agent are relative to the ablated models, so the stated relative contributions are not directly comparable.
minor comments (5)
  1. [Figure 2] The x-axis labels in Figure 2 read 'T ask Ratio'; this is a typo for 'Task Ratio'.
  2. [Table 1] The WKM row is left blank for HotpotQA without explanation; please state whether WKM is inapplicable to multi-hop QA and why.
  3. [§4.5, Algorithm 3] Algorithm 3 updates the RL-Agent with L_PG and the ML-Agent with L_FT on the few-shot support set, although in the training stage (Algorithm 2) these losses are evaluated on the query set and require LLM-generated answers and hard-negative path pools; please clarify how rewards and positive paths are obtained during few-shot adaptation.
  4. [§5.2(2), Table 2] The cross-dataset transfer experiment compares only RAP and InstructRAG; including at least one additional baseline would help substantiate the claimed 6-10% generalization improvement.
  5. [§1 and §5.2] The aggregation convention for the claimed improvements ('approximately 19.2%, 9.3%, 6.1%, and 10.2%') is not uniform: the HotpotQA, ALFWorld, and Webshop figures appear to be averages of per-backbone relative gains over RAP, while the ScienceWorld figure of 10.2% is the best single-backbone gain (GLM-4, from 24.37 to 26.85); please state the aggregation convention for each dataset.

Circularity Check

1 steps flagged · score 4.0 of 10

Partial circularity: headline numbers are selected by test-set hyperparameter sweeps, but the core advantage persists independently.

  1. fitted input called prediction [Section 5.1 Implementation Details and Section 5.2 (6)/(7), Tables 6-7]
    "Implementation Details: 'The threshold δ for constructing instruction graphs is set to 0.4.' ... 'the hyperparameter K for selecting a path is empirically set to 3.' Section 5.2(6): 'We observe that the F1 score initially increases and then decreases as δ increases.' Section 5.2(7): 'We observe that overall performance converges when K reaches 3, at which point a potentially optimal path can be retrieved from the instruction graph.'"

    The operating point used for every headline result (δ=0.4, K=3) was obtained by scanning the same F1 metric on the same reported HotpotQA test tasks and retaining the best value. Table 6 (F1 34.02→37.17) and Table 7 (F1 34.78→37.17) show that 37.17 is the maximum of the two sweeps; the abstract's 'up to 19.2% improvement' is computed from this selected value. The reported prediction is therefore an argmax over a tuning curve of the reported metric, not an independent evaluation of a fixed configuration. The circularity is partial: even the worst sweep point (34.02, K=1) beats RAP's 29.82, so the qualitative advantage is not entirely manufactured.

full rationale

The paper is an empirical system paper rather than a derivation, and most of the pipeline is not circular: the instruction graph is built from correct support-set paths, the RL-Agent and ML-Agent are trained with F1-based rewards, and the headline unseen-task results are evaluated on held-out query sets after few-shot adaptation on support sets. No self-citation chain, no imported uniqueness theorem, and no equation reduces to its own input. The one circularity-adjacent step is hyperparameter selection: δ=0.4 and K=3 are chosen from Tables 6-7 because they maximize the same F1 metric later reported as the method's performance, so the headline improvement is partly a fitted outcome. This is a partial circularity because the advantage over RAP persists across the entire δ and K sweeps. The additional concerns raised by the skeptic, namely that RAP is not the best baseline on two HotpotQA rows and that KnowAgent/RAT are omitted, are correctness and comparison-fairness issues rather than circularity, and are not scored here.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The central claim rests on three assumptions: instructions are composable, correct paths are obtainable for the support set, and meta-learned parameters transfer to unseen tasks. The free parameters delta and K are selected using the reported evaluation metric, which is a form of test-set fitting. No new entities are introduced.

free parameters (2)
  • delta (graph clustering threshold) = 0.4
    Set from Table 6 to maximize HotpotQA F1 on the unseen-task query set; no separate validation split described.
  • K (number of candidate paths) = 3
    Set from Table 7 to maximize HotpotQA F1; values 1 to 5 give 34.8 to 37.2 F1, so the choice affects the headline.
assumptions (3)
  • domain assumption Instruction composability: stored instructions can be recombined into valid new paths.
    Section 4.2: nodes group similar instructions so new paths can be formed by combining stored instructions; if instructions are entity-bound, recombination fails.
  • domain assumption Correct path availability: a correct instruction path exists and can be generated for every support-set question.
    Section 4.2 Step-1: a correct instruction path is assumed to be generated for each support question; failures are discarded, biasing the graph toward easy questions.
  • domain assumption MAML transfer: meta-training on seen tasks yields a good initialization for unseen tasks.
    Algorithms 2 and 3 assume the meta-learned parameters transfer to unseen answer types or product categories; no evidence is given for other task splits.

how reviews work

0 comments
Cite this review

Pith. "Pith review of InstructRAG: Leveraging Retrieval-Augmented Generation on Instruction Graphs for LLM-Based Task Planning." pith.science (2026). https://pith.science/paper/KARGGWTW

@misc{pith2026250413032,
  author       = {Pith},
  title        = {Pith review of: InstructRAG: Leveraging Retrieval-Augmented Generation on Instruction Graphs for LLM-Based Task Planning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KARGGWTW}},
  note         = {Machine review of arXiv:2504.13032}
}
read the original abstract

Recent advancements in large language models (LLMs) have enabled their use as agents for planning complex tasks. Existing methods typically rely on a thought-action-observation (TAO) process to enhance LLM performance, but these approaches are often constrained by the LLMs' limited knowledge of complex tasks. Retrieval-augmented generation (RAG) offers new opportunities by leveraging external databases to ground generation in retrieved information. In this paper, we identify two key challenges (enlargability and transferability) in applying RAG to task planning. We propose InstructRAG, a novel solution within a multi-agent meta-reinforcement learning framework, to address these challenges. InstructRAG includes a graph to organize past instruction paths (sequences of correct actions), an RL-Agent with Reinforcement Learning to expand graph coverage for enlargability, and an ML-Agent with Meta-Learning to improve task generalization for transferability. The two agents are trained end-to-end to optimize overall planning performance. Our experiments on four widely used task planning datasets demonstrate that InstructRAG significantly enhances performance and adapts efficiently to new tasks, achieving up to a 19.2% improvement over the best existing approach.

Figures

Figures reproduced from arXiv: 2504.13032 by the authors.

Figure 1
Figure 1. Architecture of the proposed InstructRAG with multi-agent meta-reinforcement learning, illustrated on HotpotQA. set and a query set. For support set, it is used to construct the in￾struction graph by extracting instruction paths from questions and forming the graph based on these paths. Additionally, the paths and corresponding questions help warm-start the RL-Agent, and pre-train the ML-Agent with two pre-training … view at source ↗
Figure 2
Figure 2. F1 scores and few-shot learning times wrt the number of unseen tasks or samples with DeepSeek-V2 on HotpotQA. [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. F1 scores and few-shot learning times wrt the number of unseen tasks or samples on HotpotQA, where (a)-(d) are for [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. HiPlan: Hierarchical Planning for LLM-Based Agents with Adaptive Global-Local Guidance

    cs.CL 2025-08 conditional novelty 6.0 of 10

    HiPlan melds a global milestone roadmap with per-step hints retrieved from a library of past demonstrations, and reports higher success rates than REACT, Reflexion, and TRAD on ALFWorld and WebShop.

  2. Graphs Meet AI Agents: Taxonomy, Progress, and Future Opportunities

    cs.AI 2025-06 conditional novelty 5.0 of 10

    A survey that groups graph-empowered AI agent research into planning, execution, memory, and multi-agent coordination, plus agents-for-graphs and applications.

Reference graph

Works this paper leans on

49 extracted references · 18 canonical work pages · cited by 2 Pith papers

  1. [42]

    The Talented Mr. Ripley

    Yuqi Zhu, Shuofei Qiao, Yixin Ou, Shumin Deng, Ningyu Zhang, Shiwei Lyu, Yue Shen, Lei Liang, Jinjie Gu, and Huajun Chen. 2024. Knowagent: Knowledge- augmented planning for llm-based agents. arXiv preprint arXiv:2403.03101 (2024). InstructRAG: Leveraging Retrieval-Augmented Generation on Instruction Graphs for LLM-Based Task Planning SIGIR ’25, July 13–18...

    arXiv 2024

  2. [33]

    Zihao Wang, Anji Liu, Haowei Lin, Jiaqi Li, Xiaojian Ma, and Yitao Liang. 2024. Rat: Retrieval augmented thoughts elicit context-aware reasoning in long-horizon generation. arXiv preprint arXiv:2403.05313 (2024)

    arXiv 2024

  3. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  4. [2]

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. 2024. Graph of thoughts: Solving elaborate problems with large language models. In AAAI, Vol. 38. 17682–17690

    work page 2024

  5. [3]

    Baian Chen, Chang Shu, Ehsan Shareghi, Nigel Collier, Karthik Narasimhan, and Shunyu Yao. 2023. Fireact: Toward language agent fine-tuning. arXiv preprint arXiv:2310.05915 (2023)

    arXiv 2023

  6. [4]

    Yanda Chen, Ruiqi Zhong, Sheng Zha, George Karypis, and He He. 2022. Meta- learning via Language Model In-context Tuning. In ACL. 719–730

    work page 2022

  7. [5]

    Gautier Dagan, Frank Keller, and Alex Lascarides. 2023. Dynamic planning with a llm. arXiv preprint arXiv:2308.06391 (2023)

    arXiv 2023

  8. [6]

    Budhaditya Deb, Ahmed Hassan, and Guoqing Zheng. 2022. Boosting Natural Language Generation from Instructions with Meta-Learning. In EMNLP. 6792– 6808

    work page 2022

Show all 49 references
  1. [7]

    DeepSeek-AI. 2024. DeepSeek-V2: A Strong, Economical, and Efficient Mixture- of-Experts Language Model. arXiv preprint arXiv:2405.04434 (2024)

    2024 arXiv

  2. [8]

    Chelsea Finn, Pieter Abbeel, and Sergey Levine. 2017. Model-agnostic meta- learning for fast adaptation of deep networks. In ICML. PMLR, 1126–1135

    2017

  3. [9]

    Team GLM, Aohan Zeng, Bin Xu, Bowen Wang, Chenhui Zhang, Da Yin, Diego Rojas, Guanyu Feng, Hanlin Zhao, Hanyu Lai, et al. 2024. ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools. arXiv preprint arXiv:2406.12793 (2024)

    2024 arXiv

  4. [10]

    Malte Helmert. 2006. The fast downward planning system. JAIR 26 (2006), 191–246

    2006

  5. [11]

    Tomoyuki Kagaya, Thong Jing Yuan, Yuxuan Lou, Jayashree Karlekar, Sugiri Pranata, Akira Kinose, Koki Oguri, Felix Wick, and Yang You. 2024. Rap: Retrieval- augmented planning with contextual memory for multimodal llm agents. arXiv preprint arXiv:2402.03610 (2024)

    2024 arXiv

  6. [12]

    Myeonghwa Lee, Seonho An, and Min-Soo Kim. 2024. PlanRAG: A Plan-then- Retrieval Augmented Generation for Generative Large Language Models as Decision Makers. arXiv preprint arXiv:2406.12430 (2024)

    2024 arXiv

  7. [13]

    Bo Liu, Yuqian Jiang, Xiaohan Zhang, Qiang Liu, Shiqi Zhang, Joydeep Biswas, and Peter Stone. 2023. Llm+ p: Empowering large language models with optimal planning proficiency. arXiv preprint arXiv:2304.11477 (2023)

    2023 arXiv

  8. [14]

    Jerry Liu. 2022. LlamaIndex. https://doi.org/10.5281/zenodo.1234

    2022 doi

  9. [15]

    Zhiwei Liu, Weiran Yao, Jianguo Zhang, Le Xue, Shelby Heinecke, Rithesh Murthy, Yihao Feng, Zeyuan Chen, Juan Carlos Niebles, Devansh Arpit, et al. 2023. Bolaa: Benchmarking and orchestrating llm-augmented autonomous agents. arXiv preprint arXiv:2308.05960 (2023)

    2023 arXiv

  10. [16]

    Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2024. Self-refine: Iterative refinement with self-feedback. NeurIPS 36 (2024)

    2024

  11. [17]

    Yu A Malkov and Dmitry A Yashunin. 2018. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. TPAMI 42, 4 (2018), 824–836

    2018

  12. [18]

    Sewon Min, Mike Lewis, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2021. Metaicl: Learning to learn in context. arXiv preprint arXiv:2110.15943 (2021)

    2021 arXiv

  13. [19]

    Shuofei Qiao, Runnan Fang, Ningyu Zhang, Yuqi Zhu, Xiang Chen, Shumin Deng, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. 2024. Agent Planning with World Knowledge Model. arXiv preprint arXiv:2405.14205 (2024)

    2024 arXiv

  14. [20]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In ICML. PMLR, 8748–8763

    2021

  15. [21]

    Shreyas Sundara Raman, Vanya Cohen, Eric Rosen, Ifrah Idrees, David Paulius, and Stefanie Tellex. 2022. Planning with large language models via corrective re-prompting. In NeurIPS Workshop

    2022

  16. [22]

    Zhengliang Shi, Shuo Zhang, Weiwei Sun, Shen Gao, Pengjie Ren, Zhumin Chen, and Zhaochun Ren. 2024. Generate-then-Ground in Retrieval-Augmented Gener- ation for Multi-hop Question Answering. ACL (2024)

    2024

  17. [23]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2024. Reflexion: Language agents with verbal reinforcement learning. NeurIPS 36 (2024)

    2024

  18. [24]

    Mohit Shridhar, Jesse Thomason, Daniel Gordon, Yonatan Bisk, Winson Han, Roozbeh Mottaghi, Luke Zettlemoyer, and Dieter Fox. 2020. Alfred: A benchmark for interpreting grounded instructions for everyday tasks. In Proceedings of the IEEE/CVF conference on computer vision and pa...

    2020

  19. [25]

    Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. 2020. Alfworld: Aligning text and em- bodied environments for interactive learning. arXiv preprint arXiv:2010.03768 (2020)

    2020 arXiv

  20. [26]

    Sanchit Sinha, Yuguang Yue, Victor Soto, Mayank Kulkarni, Jianhua Lu, and Aidong Zhang. 2024. MAML-en-LLM: Model agnostic meta-training of LLMs for improved in-context learning. KDD (2024)

    2024

  21. [27]

    Yifan Song, Da Yin, Xiang Yue, Jie Huang, Sujian Li, and Bill Yuchen Lin. 2024. Trial and error: Exploration-based trajectory optimization for llm agents. arXiv preprint arXiv:2403.02502 (2024)

    2024 arXiv

  22. [28]

    Lei Wang, Chen Ma, Xueyang Feng, Zeyu Zhang, Hao Yang, Jingsen Zhang, Zhiyuan Chen, Jiakai Tang, Xu Chen, Yankai Lin, et al. 2024. A survey on large language model based autonomous agents. Frontiers of Computer Science 18, 6 (2024), 186345

    2024

  23. [29]

    Ruoyao Wang, Peter Jansen, Marc-Alexandre Côté, and Prithviraj Ammanabrolu

  24. [30]

    Renxi Wang, Haonan Li, Xudong Han, Yixuan Zhang, and Timothy Baldwin

  25. [31]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171 (2022)

    2022 arXiv

  26. [32]

    Yizhong Wang, Swaroop Mishra, et al. 2022. Benchmarking generalization via in-context instructions on 1,600+ language tasks. arXiv preprint arXiv:2204.07705 2 (2022)

    2022 arXiv

  27. [34]

    Zhaowei Wang, Hongming Zhang, Tianqing Fang, Ye Tian, Yue Yang, Kaixin Ma, Xiaoman Pan, Yangqiu Song, and Dong Yu. 2024. DivScene: Benchmarking LVLMs for Object Navigation with Diverse Scenes and Objects. arXiv preprint arXiv:2410.02730 (2024)

    2024 arXiv

  28. [35]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. NeurIPS 35 (2022), 24824–24837

    2022

  29. [36]

    Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning 8, 3 (1992), 229–256

    1992

  30. [37]

    Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. 2023. The rise and potential of large language model based agents: A survey. arXiv preprint arXiv:2309.07864 (2023)

    2023 arXiv

  31. [38]

    Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. HotpotQA: A Dataset for Diverse, Explainable Multi-hop Question Answering. In EMNLP. 2369–2380

    2018

  32. [39]

    Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. 2022. Webshop: Towards scalable real-world web interaction with grounded language agents. NeurIPS 35 (2022), 20744–20757

    2022

  33. [40]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. NeurIPS 36 (2024)

    2024

  34. [41]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629 (2022)

    2022 arXiv

  35. [45]

    Sample batch of tasks

  36. [46]

    Optimize RL-Agent byL𝑊𝑆 and ML-Agent byL𝑃𝑇 on the support set for each task

  37. [47]

    Jointly optimize RL-Agent and ML-Agent byL𝑃𝐺 andL𝐹𝑇 on the query sets across all sampled tasks

  38. [48]

    Update trained RL-Agent by L𝑊𝑆 and ML-Agent byL𝑃𝑇 on the support set for each task

  39. [49]

    Search” and “Lookup

    Jointly update RL-Agent and ML-Agent byL𝑃𝐺 andL𝐹𝑇 on the support set for each task Report the average effectiveness on the query sets across all tasks Table 9: Prompt for Overall Plan on HotpotQA. Solve a question answering task with interleaving Thought, Action, Observation s...

    2025

  40. [2022]

    ScienceWorld: Is your Agent Smarter than a 5th Grader?. InEMNLP. 11279– 11298

  41. [2024]

    arXiv preprint arXiv:2402.11651 (2024)

    Learning From Failure: Integrating Negative Examples when Fine-tuning Large Language Models as Agents. arXiv preprint arXiv:2402.11651 (2024)

    2024 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.