{"id":"3067c4ac-a717-4058-8b50-51db34990de4","arxiv_id":"2509.08222","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"ExRAP couples LLM planning with a temporal knowledge-graph memory and information-based exploration, improving success and efficiency for continual embodied instruction following.","lead":"Researchers built ExRAP, a planning system that helps robots and driving agents handle many ongoing instructions in changing environments by combining a language model with a structured memory of the world. It reports higher task success and lower execution time than four LLM-based planners in three simulators.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Eq. (11)'s exploration-value proxy is unvalidated and, at the reported w_R=1.0 / w_T=0.01, it alone selects skills; the headline SR gains could depend on this miscalibrated term.","rationale":"The strongest claim is an empirical one: ExRAP outperforms LLM-Planner by 16.45% SR on average and reduces PS by 3.40. For that claim to hold as an explanation of the method, the exploration-integrated planner must actually be doing useful work. The reader's weakest assumption identifies the right soft spot: Eq. (11) is presented as following from a mild assumption, but the assumption only pins down the fully-synchronized endpoint and not the intermediate linear relationship between entropy and graph distance. The reported hyperparameters amplify the issue: with w_R=1.0 and w_T=0.01, the exploitation term contributes almost nothing, so the unvalidated exploration term is the planner. I do not see a more load-bearing concern. The leftover reviewer-response text in Appendix E.3 and the absence of public code are quality problems, and the runtime limitation is acknowledged, but they do not attack the mechanism as directly as Eq. (11). The right response is to keep the conditional verdict and request the concrete validation/ablation above.","tokens_in":20268,"tokens_out":5301,"duration_ms":61641,"concrete_test":"On the VirtualHome medium non-stationarity setting, instrument the planner to compute, for a sample of timesteps and candidate skills, the actual one-step reduction in query-evaluator entropy after executing the skill and updating the TEKG, and compare it with Eq. (11)'s predicted value via rank correlation and calibration. Then rerun ExRAP with (i) w_R=0 and (ii) w_R=1.0 but with v_R replaced by the measured MI reduction. If either change removes most of the SR gap versus LLM-Planner, the Eq. (11) proxy is load-bearing and must be validated; if the advantage persists, the concern is mitigated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.4 justifies the exploration planner with a 'mild assumption' that query-evaluator entropy reaches zero when the TEKG memory is fully synchronized. That endpoint condition does not imply Eq. (11)'s linear interpolation H(P(q|G_{t+1})) ≈ H(P(q|G_t)) · d(Φ_R(G^z_t,q))/d(Φ_R(G_t,q)) for partially updated memory, and it does not by itself justify graph distance as a proxy for mutual-information gain. Also, Eq. (10) equates a difference of mutual informations with a difference of entropies without the distributional identity that would make that step valid. This matters more than the paper acknowledges: Table A.4 sets w_R=1.0 and w_T=0.01, so the integrated score in Eq. (12) is effectively v_R alone; any miscalibration directly determines which skill the agent executes. The reported 16.45% SR gain over LLM-Planner is attributed to exactly this exploration mechanism, yet no ablation varies these weights and no comparison of Eq. (11) to an actually computed MI reduction is reported. The concern is not that the approximation is false, but that the central mechanism is unverified and load-bearing.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"ExRAP is an LLM-based planning framework for continual embodied instruction following in non-stationary environments. The agent maintains a temporal embodied knowledge graph (TEKG) as environmental memory; an instruction interpreter decomposes each continual instruction into a query and an execution; and a memory-augmented query evaluator uses retrieved TEKG quadruples to estimate query satisfaction with an entropy-based temporal consistency refinement. For planning, ExRAP combines an exploitation value (LLM in-context planning with demonstrations) with an exploration value derived from a graph-distance proxy for mutual-information reduction. The framework is evaluated in VirtualHome, ALFRED, and CARLA against ZSP, SayCan, ProgPrompt, and LLM-Planner across three non-stationarity levels, instruction scales/types, LLM sizes, and ablations. The central empirical claim is that ExRAP consistently improves success rate and reduces pending steps, e.g., 16.45% average SR gain and 3.40 PS reduction over LLM-Planner.","tokens_in":20744,"tokens_out":9919,"duration_ms":114856,"significance":"The empirical scope is a clear strength: three simulators, multiple non-stationarity levels, instruction types, instruction scales, LLM backbones, and ablation conditions, plus runtime-overhead measurement and qualitative analyses mapping ExRAP behavior to known heuristics. If the reported gains hold up, ExRAP would be a strong practical approach to continual instruction following, particularly because it maintains performance with small LLMs. However, the information-based exploration estimator that is central to the method is asserted rather than derived and is not validated against the quantity it approximates. Because Table A.4 makes this estimator the dominant term in skill selection, the empirical results do not currently establish the proposed exploration mechanism as the cause of the gains. The baselines are also author-adapted and no code is released, so independent replication is not yet possible. The manuscript would need additional validation or more modest claims before the central contribution is fully supported.","major_comments":[{"comment":"The equality \\(I(Q;G_{t+1})-I(Q;G_t)=\\sum_q H(P(q|G_t))-H(P(q|G_{t+1}))\\) does not follow from the definitions. Mutual information expands as \\(H(Q)-H(Q|G)\\); the difference equals \\(H(Q|G_t)-H(Q|G_{t+1})\\) only if the marginal \\(H(Q)\\) is invariant, and \\(H(Q|G)\\) is not \\(\\sum_q H(P(q|G))\\) unless the queries are independent Bernoulli variables. These assumptions are not stated. Eq. (11) then substitutes a graph-distance ratio for the true entropy reduction; the stated endpoint condition (entropy zero when memory is fully synchronized) does not imply the linear interpolation, and the distance function \\(d\\) is never formally defined. As written, Eqs. (10)-(11) are heuristics, not derivations.","section":"Section 3.4, Eq. (10)"},{"comment":"With \\(w_R=1.0\\) and \\(w_T=0.01\\), the argmax in Eq. (12) is effectively the exploration value \\(v_R\\) alone. Since \\(v_R\\) is defined through the same query evaluator it is meant to improve, any miscalibration in Eq. (11) directly decides which skill is executed. No ablation varies \\(w_R/w_T\\), and no experiment replaces Eq. (11) with a computed mutual-information reduction or an alternative exploration bonus. The reported 16.45% SR gain over LLM-Planner (Section 4.1) is attributed to this exploration mechanism, so the central performance claim currently rests on an unvalidated proxy. Please add weight sweeps and a validation of Eq. (11), or soften the information-based-exploration claim.","section":"Section 4 / Table A.4"},{"comment":"The temporal consistency rule \\(H(R(q|G_{t-1}))>H(P(q|G_{t-1}))\\) is an unrestricted monotonicity assumption. Enforcing it by discarding all sampled responses that violate it can systematically bias the prior when the environment is stable or when the previous estimate was overconfident; no formal justification is given. The ablation in Table 4 shows a 15.56% average SR difference, so this scheme is load-bearing for query evaluation. At minimum, report sensitivity to the refinement procedure or compare against a properly derived Bayesian decay model.","section":"Section 3.3, Eqs. (6)-(7)"}],"minor_comments":[{"comment":"Table 1 header spells 'CALRA'; it should be 'CARLA'. Appendix E heading 'Anaylsis' should be 'Analysis'. Section C.2 says 'hyperparameter settings for the baselines' but Table A.4 is for ExRAP.","section":"Table 1 / Appendix E"},{"comment":"The formula is typeset with mismatched parentheses; it should be \\(1 - d(\\Phi_R(G^z_t,\\{q\\}))/d(\\Phi_R(G_t,\\{q\\}))\\). Also, the average graph distance \\(d\\) is never defined (shortest path, embedding distance, etc.).","section":"Eq. (11)"},{"comment":"Algorithm 1 is inconsistent with Eq. (7): line 14 appends \\(P(q|G_{t-1})\\) after computing \\(R(q|G_{t-1})\\), while Eq. (7) averages the accepted \\(\\Phi_{LLM}\\) responses. Line 19 uses \\(G_{1:t-1}\\) where Eq. (5) implies \\(G_{1:t}\\).","section":"Algorithm 1"},{"comment":"ExRAP-MA appears in Table A.6 without definition or discussion in the main text; please clarify whether it is a memory-augmented variant and how it differs from ExRAP.","section":"Appendix D.2"}],"recommendation":"major_revision","confidential_remarks":"To the editor: this is a solid empirical systems paper, but the central information-theoretic component is unvalidated and is the dominant term in skill selection. I would be comfortable with acceptance after the authors either validate Eq. (11) against computed MI reduction / weight sensitivity or substantially soften the claim. The lack of code and the author-adapted baselines argue for treating the quantitative gains as preliminary. No ethical or scope concerns."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: ExRAP is worth knowing about, but don't take the exploration-value story at face value. The problem setting—continual instruction following with persistent conditional commands—is genuinely useful, and the system beats strong baselines by large margins in three simulators. The authors did real engineering: TEKG memory with contradiction-based updates, a query evaluator with temporal consistency refinement, and ablation studies that carve out the contribution of each piece. The gains over LLM-Planner (16.45% SR average) are consistent and widen with non-stationarity, which is what you'd want to see.\n\nNow the soft spots. The math in Section 3.4 is the weakest part. Eq (10) equates a difference of mutual informations with a difference of entropies without the conditional distribution identity that would make that step legal. Eq (11) then approximates the entropy after a skill as a linear interpolation using graph distance, under a 'mild assumption' that does not actually imply that interpolation. This matters because the hyperparameters in Table A.4 set w_R=1.0 and w_T=0.01, so the exploration value alone decides the skill. The paper attributes the headline gains to exactly this mechanism, yet it never varies those weights and never checks Eq (11) against a computed MI value. The exploration planner could be a fine heuristic—the qualitative analysis suggests it behaves like a staleness-aware policy—but the MI framing is not validated.\n\nAlso there are editorial issues: 'As the reviewer suggested' appears in Appendix E.3, and an ablation variant ExRAP-MA shows up in Table A.6 with no definition. No code release either, so the baseline adaptations (e.g., step-wise LLM-Planner) can't be checked independently.\n\nIs the central claim undercut? I don't think so. The performance numbers are strong and the ablations show the exploration component matters. But the reason it matters is under-specified. The paper would be stronger if it reframed Eq (11) as a heuristic and added a sensitivity analysis over w_R and w_T, or measured actual information gain on a few trajectories.\n\nWho's this for? Anyone working on LLM-based embodied planning, RAG for robots, or memory-augmented reasoning. It deserves a serious referee—send it out—but the authors should be pushed to justify or reframe the exploration estimate and to clean up the manuscript.","headline":"A strong empirical system for a genuinely new problem, but the exploration-value approximation is asserted rather than justified—worth peer review, not blind trust.","tokens_in":21048,"tokens_out":2370,"would_cite":true,"duration_ms":28097,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By coupling an LLM planner with a temporal knowledge-graph memory and exploration priced by information gain, ExRAP claims a 16.45% average success-rate gain and 3.40 fewer pending steps than LLM-Planner on VirtualHome, ALFRED, and CARLA.","keywords":["continual instruction following","embodied task planning","retrieval-augmented planning","temporal knowledge graph","non-stationary environments","exploration-exploitation balance","LLM-based agents"],"falsifier":"Log, for each skill in a controlled VirtualHome run, the exploration value predicted by Eq. (11) against the entropy drop actually measured in the query evaluator after the skill's observation; if high-predicted-value skills consistently yield near-zero entropy drops, the graph-distance proxy does not track information gain. A second check: in a static environment, compare ExRAP against the exploitation-only variant ExRAP-EXP — if the success gap vanishes, the exploration term's value rests entirely on the decay assumption.","tokens_in":20260,"feed_emoji":"🤖","tokens_out":10566,"duration_ms":99290,"temperature":0.7,"pith_summary":"This paper targets a practical gap in embodied AI: instructions that arrive continuously — 'if the temperature is high, open the window' — in environments that keep changing, where a one-shot plan is never enough. ExRAP gives a large language model a two-sided job: answer conditional queries against a temporal knowledge-graph memory of the world, and choose skills that both complete tasks and deliberately reduce uncertainty about stale knowledge. The paper's central argument is that exploration should be priced by expected information gain, not left to the LLM's intuition, and that query confidence should decay as observations age. In VirtualHome, ALFRED, and CARLA, ExRAP reports higher success rates and fewer pending steps than ZSP, SayCan, ProgPrompt, and LLM-Planner across every tested level of non-stationarity. If the results hold, the framework points toward agents that handle ongoing user requests in real homes and roads without re-collecting the whole world state at each decision.","feed_headline":"Memory-guided exploration boosts embodied instruction success by 16%","feed_subtitle":"ExRAP pairs LLM planning with a temporal knowledge graph so agents keep up as homes and roads change.","key_machinery":"The load-bearing object is the temporal embodied knowledge graph (TEKG), a growing set of (source entity, relation, target entity, timestamp) quadruples that serve as the agent's environmental context memory. Retrieval from the TEKG feeds both the query evaluator and the exploitation planner, so one memory grounds condition-checking and skill choice. On top of it sits the exploration value vR(Gt,z) = sum over queries of H(P(q|Gt)) times [1 − d(Phi_R(Gz_t, {q})) / d(Phi_R(Gt, {q}))], which converts 'how much follow-up uncertainty will this skill remove' into a number the planner maximizes alongside the LLM's exploitation score. This proxy formula is the mechanism that balances keeping memory","core_discovery":"ExRAP's central claim is that the bottleneck in continual embodied instruction following is memory freshness, not plan generation: when conditions change over time, an agent must know when its picture of the world is stale. The paper splits each instruction into a query against the environment plus an execution that fires when the query is satisfied, stores the world as a temporal embodied knowledge graph, and has an LLM evaluate queries against that graph while an exploration planner prices skills by how much they shrink query uncertainty. A temporal-consistency step forces evaluator confidence to fall as observations age. Across VirtualHome, ALFRED, and CARLA, ExRAP reports a 16.45% averag","pith_inferences":["Because the exploration value uses graph distance as a stand-in for uncertainty reduction, a natural extension is metric or continuous-state environments (object positions, temperatures, moving vehicles), where the distance proxy would need re-derivation and could be validated directly against measured entropy drops.","The temporal-consistency rule — confidence must fall as memory ages — may misfire in genuinely periodic environments, where old observations of a regularly toggled state still carry information; per-query decay rates would be a testable refinement.","The reported weights (exploration value 1.0, exploitation value 0.01) make ExRAP heavily exploration-biased by construction; tuning this ratio per environment or per instruction set is an untested lever that could trade steps for success differently.","Robustness to a small backbone hints that the memory and exploration machinery, not the language model, does most of the work; a deployment-relevant test would run ExRAP on a robot with fixed compute and measure throughput against the reported retrieval speedup."],"forward_implications":["Continual instructions can be handled as persistent queries over a knowledge-graph memory instead of re-planned from scratch at each step; this is why ExRAP's edge over LLM-Planner widens as the number of simultaneous instructions grows, from an 18.49-point SR gap at small scale to 22.04 points at large scale.","Temporal consistency is doing real work: dropping it lowers success by 15.56% on average in the ablation, so modeling knowledge decay is part of what keeps query evaluation reliable.","Pricing exploration by information gain pays off most when the world changes fastest: the reported SR gap over LLM-Planner widens from 15.35 points at low non-stationarity to 18.58 points at high non-stationarity.","The memory-plus-exploration design, not a simulator-specific trick, carries the result: the same framework reports consistent gains in household (VirtualHome, ALFRED) and driving (CARLA) domains.","The approach tolerates smaller language models: with Gemma-2B, ExRAP holds about 52.75% success where LLM-Planner drops to about 23.31%, suggesting memory compensation reduces dependence on model scale."],"supporting_citations":[{"why":"The strongest baseline: a step-wise LLM planner built on retrieved demonstrations; the reported SR and PS gains are measured against it.","marker":"[29]"},{"why":"Source of the mutual-information exploration idea that ExRAP adapts into its exploration planner.","marker":"[17]"},{"why":"The database notion of continuous queries that motivates framing instructions as persistent condition queries.","marker":"[18]"},{"why":"VirtualHome, the household simulator where the main results and ablations run.","marker":"[24]"},{"why":"ALFRED, the vision-and-language household benchmark used as a second evaluation domain.","marker":"[26]"},{"why":"CARLA, the driving simulator used to show the framework transfers beyond the home.","marker":"[7]"},{"why":"SayCan, a baseline pairing LLM language scores with learned affordance values.","marker":"[4]"},{"why":"ProgPrompt, the code-style prompting baseline that competes on plan generation.","marker":"[28]"},{"why":"ZSP, the zero-shot LLM planner used as the weakest planning baseline.","marker":"[12]"}],"fun_headline_variants":["Robots refresh aging memory to follow changing instructions","Explore to remember: ExRAP boosts embodied task success by 16%","When worlds change, robots must explore to plan better","Fresh memory beats smarter planning in changing homes","Exploratory retrieval + temporal graphs = 16% better robot plans"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The exploration planner's value rests on the assumption that the graph distance between the agent and the retrieved knowledge accurately measures how much uncertainty a skill will remove, and that fully synchronized memory means zero query uncertainty; if that proxy is miscalibrated, exploration effort goes to the wrong places and the reported gains shrink.","fun_headline_variants_meta":{"raw":{"variants":["Robots refresh aging memory to follow changing instructions","Explore to remember: ExRAP boosts embodied task success by 16%","When worlds change, robots must explore to plan better","Fresh memory beats smarter planning in changing homes","Exploratory retrieval + temporal graphs = 16% better robot plans"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00051,"raw_usage":{"total_tokens":2337,"prompt_tokens":777,"completion_tokens":1560,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":1478}},"tokens_in":521,"tokens_out":1560,"duration_ms":14658,"temperature":1.0,"reasoning_tokens":1478,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T21:00:18.358570+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Log, for each skill in a controlled VirtualHome run, the exploration value predicted by Eq. (11) against the entropy drop actually measured in the query evaluator after the skill's observation; if high-predicted-value skills consistently yield near-zero entropy drops, the graph-distance proxy does not track information gain. A second check: in a static environment, compare ExRAP against the exploitation-only variant ExRAP-EXP — if the success gap vanishes, the exploration term's value rests entirely on the decay assumption.","supporting_citations":[],"review_version":1}