{"id":"d87d314d-9f1a-4974-becd-7bb94f7be3e9","arxiv_id":"2504.13032","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"InstructRAG combines instruction paths from a graph via reinforcement learning and meta-learned selection, reporting relative gains of 2 to 25 percent over its chosen baselines across four LLM planning benchmarks.","lead":"InstructRAG stores successful action sequences from past tasks in a graph and uses two learned agents to combine and retrieve them as examples for a large language model. The authors report consistent gains over earlier retrieval-based planners on four task-planning benchmarks, though no code or data is released.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline '19.2% improvement over the best existing approach' is computed against RAP, which is not the best baseline on two of three HotpotQA rows, and the most relevant TAO/RAG baselines are omitted.","rationale":"The reader's verdict is CONDITIONAL, and the reader's rationale already lists 'missing strong baselines' as a reason for conditionality. However, the reader's stated weakest assumption is about instruction modularity, whereas the most load-bearing concern for the central claim is the baseline-selection and headline-metric issue. The abstract's '19.2% improvement over the best existing approach' is the sentence that carries the paper's central contribution, and it is directly contradicted by the table the authors provide: on two of three HotpotQA rows the best baseline is GenGround, not RAP, so the aggregate number can only be obtained by comparing to a baseline that is not the best. Adding KnowAgent and RAT, which are the methods most comparable to the proposed TAO/RAG pipeline, is the single check that would settle whether the claimed advantage holds against the actual state of the art. This does not change the verdict: the paper should still be accepted only if the authors provide the missing comparisons, artifacts, and a corrected headline statistic. I therefore keep the reader's CONDITIONAL verdict unchanged, with partial agreement because the reader identified the concern but did not make it the primary weakest assumption.","tokens_in":25155,"tokens_out":15034,"duration_ms":143645,"concrete_test":"Rerun Table 1 with KnowAgent and RAT under identical settings (same three backbone LLMs, same support/query splits, same retrieval data) and recompute every reported improvement against the actual highest-scoring baseline in each (dataset, backbone) cell. If the average relative gain over the true best baseline falls below the claimed 19.2%, or if KnowAgent or RAT matches InstructRAG on HotpotQA, the central claim of superiority over the best existing approach is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central quantitative claim is the '19.2% improvement over the best existing approach' in the abstract and Section 5.2. This number is not reproducible from Table 1 in the way claimed. The reported value matches the average of per-backbone relative improvements over RAP on HotpotQA, but RAP is not the best baseline on two of the three HotpotQA rows: with GPT-4o mini, GenGround scores 28.99 while RAP scores 27.66, and with DeepSeek-V2, GenGround scores 33.50 while RAP scores 29.82. Against the actual per-cell best baseline, the average HotpotQA improvement is roughly 12-14%, not 19.2%. In addition, the two strongest related approaches for this setting, KnowAgent and RAT, are not included as baselines even though they are discussed in Section 2 and are directly relevant to TAO-style RAG planning. The statement 'improves performance by approximately 19.2%, 9.3%, 6.1%, and 10.2% over the best baseline method' therefore conflates a specific comparison to RAP with a comparison to the best existing approach. This makes the headline superiority claim unverified, not because the method is internally inconsistent, but because the evidence table does not establish that the compared baseline is the best available.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper addresses retrieval-augmented generation (RAG) for LLM-based task planning and identifies two challenges: enlargability (recombining stored instruction paths to cover new questions within a task family) and transferability (rapid adaptation to unseen tasks). The proposed InstructRAG builds a directed instruction graph from correct instruction paths extracted on a support set, trains an RL-Agent (REINFORCE with a warm-start phase) to traverse the graph and retrieve K candidate paths, and trains an ML-Agent (contrastive pre-training plus hard-negative fine-tuning) to select one path for use as an in-context exemplar in the prompt to the planning LLM; the two agents are optimized jointly in a MAML-style outer loop. Experiments on HotpotQA, ALFWorld, Webshop, and ScienceWorld with GLM-4, GPT-4o mini, and DeepSeek-V2 report consistent gains over ReAct, WKM, Reflexion, GenGround, and RAP, together with ablations, a noise-robustness study, parameter studies for the clustering threshold delta and the candidate-path count K, and few-shot adaptation curves.","tokens_in":25445,"tokens_out":10739,"duration_ms":93876,"significance":"If the empirical claims hold, the paper makes a useful contribution: it formalizes instruction-level reuse in RAG planning, and its component-wise ablations (Table 5), robustness to noisy historical paths (Table 4), cross-dataset transfer evidence (Table 2), and detailed qualitative trajectory comparisons (Tables 12-14) are well-conceived and largely support the internal logic of the design. The complete listings of Algorithms 1-4 and the explicit meta-reinforcement-learning training recipe make the method concretely reproducible in principle, and the few-shot adaptation experiments (Figure 2) directly probe the transferability property that the paper defines. The significance is tempered, however, by the way the headline improvement is computed and by hyperparameter selection on the reported test metric; the current evidence supports 'competitive with, and often better than, the evaluated baselines', but not yet the abstract's 'best existing approach' claim.","major_comments":[{"comment":"The headline claim of 'up to a 19.2% improvement over the best existing approach' is not supported by Table 1 as reported. The 19.2% HotpotQA figure is computed against RAP, but RAP is not the best baseline in two of the three HotpotQA rows: with GPT-4o mini, GenGround achieves 28.99 versus RAP's 27.66, and with DeepSeek-V2, GenGround achieves 33.50 versus RAP's 29.82. Against the per-cell best baseline, the average HotpotQA improvement is roughly 7-11%, not 19.2%, so the phrase in §5.2(1), 'over the best baseline (RAP)', is internally inconsistent with the table. In addition, KnowAgent [42] and RAT [33], both discussed in Section 2 as directly relevant TAO-style planning methods (with RAT being RAG-based), are not evaluated, and the GenGround baseline is implemented as a LlamaIndex retriever over stored TAO triplets rather than its published generate-then-ground procedure. The authors should add the missing baselines or explicitly rescope the claim to the evaluated baseline set.","section":"Abstract; §5.2(1), Table 1; §2"},{"comment":"The hyperparameters delta=0.4 and K=3 are selected from Tables 6-7, which report the same HotpotQA F1 metric that constitutes the paper's headline result, and no validation split or hyperparameter-selection protocol is described in the experimental setup. This means the reported headline numbers partly reflect test-set fitting. The authors should either select delta and K on a held-out validation split and report the corresponding test numbers, or show that the advantage over baselines is robust to alternative settings; notably, Table 7 already shows that even K=1 (F1 34.78 on DeepSeek-V2) beats GenGround (33.50), so a validation-based selection is likely to preserve the main conclusion, but the current reporting convention is methodologically problematic.","section":"§5.1 Implementation Details; §5.2(6)-(7), Tables 6-7"},{"comment":"The statement that 'all reported experimental results are statistically significant, verified by a t-test with p < 0.05' cannot be checked from the paper: the tables report a single F1, success-rate, or reward number per method and backbone, with no indication of the number of runs, the variance, or whether the test was paired per task or per question. Please specify the test design and report the underlying dispersion (for example, per-task scores or standard deviations across runs). In addition, the ablation attribution in Section 5.2(5) uses inconsistent denominators: the 11.6% attributed to removing the instruction graph is relative to the full model, while the 11.1% and 6.9% attributed to removing the RL-Agent and ML-Agent are relative to the ablated models, so the stated relative contributions are not directly comparable.","section":"§5.1 Evaluation Metrics; §5.2(5), Table 5"}],"minor_comments":[{"comment":"The x-axis labels in Figure 2 read 'T ask Ratio'; this is a typo for 'Task Ratio'.","section":"Figure 2"},{"comment":"The WKM row is left blank for HotpotQA without explanation; please state whether WKM is inapplicable to multi-hop QA and why.","section":"Table 1"},{"comment":"Algorithm 3 updates the RL-Agent with L_PG and the ML-Agent with L_FT on the few-shot support set, although in the training stage (Algorithm 2) these losses are evaluated on the query set and require LLM-generated answers and hard-negative path pools; please clarify how rewards and positive paths are obtained during few-shot adaptation.","section":"§4.5, Algorithm 3"},{"comment":"The cross-dataset transfer experiment compares only RAP and InstructRAG; including at least one additional baseline would help substantiate the claimed 6-10% generalization improvement.","section":"§5.2(2), Table 2"},{"comment":"The aggregation convention for the claimed improvements ('approximately 19.2%, 9.3%, 6.1%, and 10.2%') is not uniform: the HotpotQA, ALFWorld, and Webshop figures appear to be averages of per-backbone relative gains over RAP, while the ScienceWorld figure of 10.2% is the best single-backbone gain (GLM-4, from 24.37 to 26.85); please state the aggregation convention for each dataset.","section":"§1 and §5.2"}],"recommendation":"major_revision","confidential_remarks":"For the editor: the central mechanism of the paper is interesting and the internal ablations are mostly convincing, but the empirical presentation overstates the result in two fixable ways: the headline comparison is made against RAP rather than the per-cell best baseline and omits the directly relevant KnowAgent and RAT methods, and the hyperparameters delta and K appear to be tuned on the reported test metric without a validation split. Both issues are addressable within the scope of a revision, as is the unverifiable statistical-significance sentence. I would also ask that the authors be encouraged to release code or trajectory data, since the LLM-in-the-loop training procedure would otherwise be costly to reproduce."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the instruction graph is a real idea and the paper is worth a referee's time, but the headline number is not computed against the best baseline and two key hyperparameters are tuned on the test metric. Treat the 19.2% claim as unverified.\n\nThe genuinely new piece is organizing past instruction paths as a graph, clustering similar instructions into junction nodes, and letting a learned RL policy traverse the graph to combine fragments into new paths. That is a plausible mechanism for improving coverage in TAO-style planning, and it is distinct from RAP's flat memory and GenGround's grounding. The two-agent setup (RL for path construction, ML for path selection) is a reasonable division and the ablations suggest both parts matter. Running the method across three backbones and four datasets, and reporting gains over ReAct, Reflexion, RAP, GenGround, and WKM, gives the empirical claim some weight.\n\nNow the soft spots. The abstract says \"19.2% improvement over the best existing approach.\" That number is the average per-backbone improvement over RAP on HotpotQA. But on the GPT-4o mini and DeepSeek-V2 rows, GenGround beats RAP. Against the per-cell best baseline, the average improvement is roughly 12-14%, which is still respectable but not 19.2%. The claim should be corrected. Second, delta and K are set by looking at F1 on the same test set used for reporting (Tables 6-7); there is no visible validation split. That is a form of test-set fitting and it inflates the result. Third, KnowAgent and RAT are discussed in the related work and are directly relevant, but they are not in the baseline table. Omitting them makes the \"best existing approach\" comparison incomplete. Fourth, no code or data is released, which is a problem for a paper whose value is empirical. The t-test significance claim would also be easier to judge with standard deviations.\n\nNone of these issues kills the idea. They are fixable: release artifacts, add a validation split for the hyperparameters, include the missing baselines, and recompute the headline. If those changes hold up, the instruction graph could be a useful contribution for people building RAG-based planners.\n\nWho should read this: researchers working on retrieval-augmented generation for LLM agents, especially those using TAO processes. It is not a paradigm shift, but it offers a concrete, testable mechanism for expanding planning knowledge from stored experiences.\n\nRecommendation: yes, send it to a serious referee. The core mechanism deserves scrutiny, and the empirical questions raised here are exactly the kind that should be resolved in review, not by desk rejection.","headline":"Instruction graph is a genuine new mechanism, but the headline improvement is overstated and hyperparameters are tuned on the test set.","tokens_in":25952,"tokens_out":3389,"would_cite":true,"duration_ms":31223,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"InstructRAG claims that storing past planning actions as a graph of reusable instruction nodes and learning to recombine them with reinforcement and meta-learning improves LLM task planning by up to 19.2% over the best baseline.","keywords":["large language model","retrieval-augmented generation","agent planning","instruction graph","meta-reinforcement learning","enlargability","transferability","few-shot learning"],"falsifier":"On a HotpotQA test split where no question shares any entity with any instruction stored in the graph, if InstructRAG's F1 drops to the level of whole-trajectory retrieval, such as RAP, then the graph's recombination is not what produces new valid plans, and the enlargability claim would be falsified.","tokens_in":24970,"feed_emoji":"🧩","tokens_out":9403,"duration_ms":88990,"temperature":0.7,"pith_summary":"The paper argues that retrieval-augmented generation for LLM task planning should retrieve at the level of individual instruction steps rather than whole trajectories. It identifies two properties a planning RAG needs: enlargability, the ability to answer new questions by recombining previously stored instructions, and transferability, the ability to adapt quickly to entirely new tasks. To get both, InstructRAG builds an instruction graph from successful past plans, trains an RL-Agent to traverse the graph and propose candidate instruction paths, and trains an ML-Agent to meta-learn which candidate path will best help the LLM answer. The two agents are trained end-to-end so that the LLM's final answer score rewards path selection. If the central claim is right, a planning agent can do better by reorganizing its memory as reusable action components than by retrieving full past experiences, with reported gains as large as 19.2% over the best comparison method.","feed_headline":"Stored instructions, recombined, lift LLM planning by 19.2%","feed_subtitle":"Reusable action steps plus learned path selection beat whole-trajectory retrieval on four planning benchmarks.","key_machinery":"The load-bearing object is the instruction graph, a directed graph in which each node is a set of similar instructions (merged by an AKNN similarity check against a threshold δ) and each edge is a set of tasks and questions connected by those instructions. Adjacent instructions are never placed in the same node, and similar instructions from different tasks can merge into one node, forming junction nodes where new paths can be spliced together. On top of the graph, an RL-Agent conducts depth-first traversal, deciding include or exclude at each node from a state of three cosine similarities between the question and the node's instructions, edge tasks, and exemplar questions, and receives the LLM's end-to-end metric, such as F1, as reward. An ML-Agent then ranks the K candidate paths by learned question-path similarity, builds the prompt, and is adapted to new tasks with MAML-style meta-updates. The graph provides compositionality, while the two agents provide learned path search and learned prompt selection.","core_discovery":"On the author's own terms, the discovery is that a RAG planner is materially improved when its external database is an instruction graph—nodes cluster similar atomic instructions such as Search[entity] and Lookup[keyword], edges record the tasks those instructions were part of—because the graph turns stored plans into composable parts. The RL-Agent treats traversal as an MDP and learns whether to include or exclude each node, retrieving K candidate paths; the ML-Agent uses contrastive question-path alignment plus meta-learning to pick the single path as an in-context exemplar. Together they explicitly optimize the two properties the paper names: enlargability (new questions answered by combining stored instructions) and transferability (few-shot adaptation to unseen tasks). Across HotpotQA, ALFWorld, Webshop, and ScienceWorld, and over three backbone LLMs, the system reports consistent gains over RAP, the strongest comparison, including 19.2% on HotpotQA with DeepSeek-V2.","pith_inferences":["Editorial inference: the junction-node mechanism treats the action vocabulary as a small set of reusable operators over a large entity space; if that is the real source of gain, the same graph recombination should transfer to other plan domains with compositional action vocabularies, such as tool-use coding or robot manipulation.","Editorial inference: because the RL reward is the answer score of one specific LLM, the learned path policy is coupled to that LLM; porting the trained agents to a new backbone would likely require re-running the reward loop rather than reusing precomputed paths.","Editorial inference: the threshold δ controls how aggressively instructions merge into junction nodes, and the paper fixes it at 0.4; an adaptive per-node threshold that reflects how often a node is reused across tasks is a direct, testable extension that could either sharpen or bound the enlargability gain."],"forward_implications":["Organizing past experience as a graph of composable instruction steps, instead of retrieving whole trajectories, is what drives the reported gains: removing the graph costs 11.6% on HotpotQA, removing the RL-Agent costs 11.1%, and removing the ML-Agent costs 6.9%.","The improvements are not limited to the training distribution: InstructRAG beats RAP on unseen HotpotQA, ALFWorld, and Webshop tasks, and a HotpotQA-trained model generalizes to ScienceWorld with 6–10% gains.","The same relative gains appear with three different backbone LLMs, so the mechanism is not tied to one model's reasoning style.","A RAG planner built this way tolerates polluted memory: injecting failed instruction paths up to a 50% noise rate degrades F1 by only 11.1%.","Few-shot adaptation to new tasks is fast in practice: per-task adaptation takes about 27.1 minutes and performance plateaus once about 80% of support samples are used."],"supporting_citations":[{"why":"HotpotQA supplies the multi-hop QA task, the Search/Lookup action vocabulary, and the F1 metric that serves as the end-to-end reward.","marker":"[38]"},{"why":"ReAct defines the thought-action-observation process and the action format that instruction paths are built from, and is also a baseline.","marker":"[41]"},{"why":"RAP is the strongest comparison baseline; its whole-trajectory memory retrieval is the approach InstructRAG is designed to beat.","marker":"[11]"},{"why":"GenGround is the generate-then-ground RAG baseline for multi-hop reasoning that InstructRAG compares against.","marker":"[22]"},{"why":"KnowAgent supplies the predefined action-transition rules, such as Search before Lookup, that the RL-Agent can incorporate, and its fine-tuning recipe is followed for trainable LLMs.","marker":"[42]"},{"why":"MAML provides the model-agnostic meta-learning update scheme used to adapt both agents to new tasks with few shots.","marker":"[8]"},{"why":"MetaICL is the evidence cited for the claim that meta-learning improves in-context learning generalization, which motivates the ML-Agent design.","marker":"[18]"},{"why":"ALFWorld provides the embodied household-planning benchmark and the success-rate metric reported in the experiments.","marker":"[25]"},{"why":"Webshop provides the web-shopping planning benchmark and the reward-score metric used for comparison.","marker":"[39]"},{"why":"ScienceWorld is the cross-dataset generalization test on which InstructRAG is evaluated against RAP after training on HotpotQA.","marker":"[29]"}],"fun_headline_variants":["Instruction graph boosts LLM task planning by 19.2%","RL and meta-learning on an instruction graph lift planning accuracy","Recombinable instruction paths improve LLM planning by 19.2%","InstructRAG: Graph-based retrieval enhances LLM planning by 19.2%","Composable instruction graphs raise LLM planning up to 19.2%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that individual instructions like Search[entity], Lookup[keyword], and GoTo[room] are reusable building blocks, so splicing stored instructions into a new sequence yields a valid plan even when the exact combination of entities in the new question was never seen together in any stored path.","fun_headline_variants_meta":{"raw":{"variants":["Instruction graph boosts LLM task planning by 19.2%","RL and meta-learning on an instruction graph lift planning accuracy","Recombinable instruction paths improve LLM planning by 19.2%","InstructRAG: Graph-based retrieval enhances LLM planning by 19.2%","Composable instruction graphs raise LLM planning up to 19.2%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000391,"raw_usage":{"total_tokens":2065,"prompt_tokens":959,"completion_tokens":1106,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":575,"completion_tokens_details":{"reasoning_tokens":1007}},"tokens_in":575,"tokens_out":1106,"duration_ms":7230,"temperature":1.0,"reasoning_tokens":1007,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T12:16:22.761483+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a HotpotQA test split where no question shares any entity with any instruction stored in the graph, if InstructRAG's F1 drops to the level of whole-trajectory retrieval, such as RAP, then the graph's recombination is not what produces new valid plans, and the enlargability claim would be falsified.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"HotpotQA supplies the multi-hop QA task, the Search/Lookup action vocabulary, and the F1 metric that serves as the end-to-end reward."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GenGround is the generate-then-ground RAG baseline for multi-hop reasoning that InstructRAG compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"MAML provides the model-agnostic meta-learning update scheme used to adapt both agents to new tasks with few shots."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Webshop provides the web-shopping planning benchmark and the reward-score metric used for comparison."}],"review_version":1}