{"id":"c0045d95-ebf5-470a-a2f1-69b8879446cb","arxiv_id":"2608.08523","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A quality-diversity archive of planning-policy templates with stall-triggered switching to behaviorally distant policies improves success on the ThreeDWorld transport benchmark.","lead":"This paper proposes an archive of diverse planning policies for embodied agents, built offline with quality-diversity search and used online to switch strategies when progress stalls. The idea is that behavioral variety, not a single smarter prompt, makes long-horizon planning more robust.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1's gains are point estimates with no stated separation between archive-construction rollouts and evaluation episodes, so the headline improvement could be selection on the test set rather than evidence for QD-based robustness.","rationale":"The reader's weakest assumption concerns the behavior descriptors. That is a genuine design risk, but Table 4 partially addresses it: replacing behavior-aware selection with quality-only/random selection reduces total success from 0.47 to 0.41, which is evidence that descriptor-based selection contributes. No ablation, however, can repair a confounded evaluation. If the archive was built and elite-selected on the same episodes reported in Table 1, the comparison against RHP/CoELA is not a fair test of any mechanism, diversity-based or otherwise. The manuscript's own text points to the missing details: Section 4.1 promises 'Further experimental details are provided in the Appendix,' and the appendix is absent from the arXiv version. Since code is released, the split question is checkable; until it is checked and variance is reported, the empirical claim is best treated as unverified rather than conditionally accepted.","tokens_in":12144,"tokens_out":10354,"duration_ms":121259,"concrete_test":"Open the released repository (github.com/EvoNexusX/2026XuQD-Plan.git) and trace the evaluation harness for Table 1. Check whether the episode IDs used by Algorithm 1 for offline rollouts, experience-buffer updates, and archive elite selection intersect the episode IDs used to compute the reported success rates. If they intersect, rerun the full pipeline with a disjoint held-out split; if they are already disjoint, rerun Table 1 with at least 10 independent seeds or episode-level bootstrap resampling and report 95% confidence intervals. If the gains disappear or include zero, the central claim is unsupported; if they persist with disjoint splits and tight intervals, the claim is substantially strengthened.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"The central empirical claim is that Ours w/ QD (GPT-4) improves food success from 0.33 (RHP) and 0.42 (CoELA) to 0.51, and stuff from 0.24/0.36 to 0.43, under the same 2000-step budget (Section 4.2, Table 1). For this comparison to support the diversity-driven robustness claim, the episodes used to build the archive must be disjoint from those scored in the table. The manuscript never states this. In Section 3.3.2, Algorithm 1 rolls out every offspring, uses the rollouts to compute Q(tau) (Eq. 10) and BD(tau) (Eqs. 4-5), updates the experience buffer, and selects archive elites (Eq. 13). Section 4.1 only says 'standard food and stuff splits' and defers to an appendix that is not included in the preprint. If those rollouts use the same episodes as the reported evaluation, then the QD pipeline has effectively been optimized on the test set, whereas RHP and CoELA have not; the reported 0.18/0.19 gains would be an artifact of evaluation-set selection, not a demonstration that diverse policies help. Even with a disjoint split, the table gives only point estimates with no seeds or episode-level confidence intervals, so the magnitude of the improvement is currently unquantified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a Quality-Diversity (QD) framework for multimodal embodied planning. Offline, planning-policy templates are evolved with a MAP-Elites-style search and organized into an archive indexed by two hand-defined behavior descriptors (interaction intensity and goal-directedness, Eqs. 4 and 5). Online, the agent executes a single policy, detects stalls via a fixed threshold on progress-state stagnation (Eq. 15), rolls back to a checkpoint, and switches to the behaviorally farthest untried archive elite (Eq. 17). Experiments on TDW-MAT report improved food/stuff success over RHP and CoELA baselines, and a small VLN-CE pilot is used to argue that the recovery mechanism transfers across domains. The central empirical claims are that the full method raises Food success from 0.33 (RHP) and 0.42 (CoELA) to 0.51, and Stuff from 0.24/0.36 to 0.43, under a 2000-step budget.","tokens_in":12439,"tokens_out":5608,"duration_ms":57179,"significance":"If the empirical claims were well supported, the paper would make a useful contribution: it applies QD search to LLM prompt-level planning policies in a way that is clearly described, releases code, and includes an ablation separating online switching from archive selection. The online rollback-and-switch mechanism is a sensible response to behavioral lock-in, and the idea of treating behavioral diversity as a recoverable resource is worth exploring. However, the current evaluation does not yet establish the central claim. The absence of any statement that archive-construction rollouts are disjoint from evaluation episodes, the lack of error bars or significance tests, and the underspecified ablations all prevent verification of the reported gains. The VLN-CE pilot is too thinly described to support the cross-domain transfer claim. With additional experiments and clarification, the core idea may be salvageable, but the evidence as presented is not sufficient.","major_comments":[{"comment":"The manuscript never states that the episodes used for offline rollout evaluation (Algorithm 1, line 14, which produces Q(τ) in Eq. (10) and BD(τ) in Eqs. (4)–(5)) are disjoint from the episodes scored in Tables 1, 2, and 4. If the archive is constructed on the same 'standard food and stuff splits' used for evaluation, then the QD pipeline has been effectively selected on the test set, whereas RHP and CoELA have not; the headline improvements (e.g., 0.33 to 0.51 on Food, 0.24 to 0.43 on Stuff) would then be an artifact of evaluation-set selection rather than evidence for diversity-driven robustness. Please specify the exact episode split between offline rollouts and online evaluation, and if no such split exists, re-run with a disjoint held-out split.","section":"Section 3.3.2, Algorithm 1; Section 4.1"},{"comment":"All reported successes are point estimates with no seeds, standard deviations, or significance tests. For example, the VLN-CE SR increase from 35.2% to 37.3% (Table 3) and the Food gain from 0.42 (CoELA) to 0.51 (Ours) in Table 1 are within the range of plausible noise for a single run. Without multiple independent runs or episode-level bootstrap confidence intervals, the magnitude of the claimed improvement is unquantified, and statements such as 'the improvement is consistent across both task categories' (Section 4.2) are not supportable. Please report variance or confidence intervals for all main tables.","section":"Section 4.1, Tables 1–4"},{"comment":"The ablation 'Ours (no OS)' is underspecified: when stall-based switching is disabled, it is unclear which policy is followed for the entire episode—the highest-quality archive elite, a random archive entry, or the initial seed template. The 'OS + Abl. ArchSel' condition is also ambiguous because it is described as relying on 'a simplified criterion (quality-only, or an equivalent random elite choice)'; these two choices can give very different baselines. Without a precise selection rule, the ablation cannot support the causal claims in Section 4.3 that online switching and behavior-aware selection each contribute to the observed gains.","section":"Section 4.3, Table 4"},{"comment":"The VLN-CE pilot is presented as evidence that the recovery mechanism transfers across domains, but the manuscript does not describe how the R2R recovery archive is constructed (behavior-descriptor definitions, stall threshold, archive size, or how the 'separate R2R archive briefly supplies' a policy is controlled), and no variance is reported. With only a +2.1 SR and −0.16 NE difference on an 11-scene subset, and no statistical support, this pilot does not currently substantiate the cross-domain transfer claim. Please either provide a full protocol and significance testing or explicitly recharacterize the pilot as illustrative rather than evidentiary.","section":"Section 4.2, Table 3"}],"minor_comments":[{"comment":"The text says 'Further experimental details are provided in the Appendix,' but no appendix is present in the manuscript; please include it or state the missing hyperparameters (population size, p_a, p_m, p_r, α, iteration budget T, archive size) in the main text.","section":"Section 4.1"},{"comment":"The caption does not explain the difference between panels (a) and (b), and the behavior-grid axes are not labeled; please clarify what each panel shows.","section":"Figure 3"},{"comment":"The behavior descriptors in Eqs. (4)–(5) depend on action categories such as 'interaction-oriented,' 'milestone-advancing,' and 'repeated low-value actions,' but the manuscript does not define how these categories are determined from raw trajectories; please provide the classification rule.","section":"Section 3.3.1"},{"comment":"The phrase 'standard food and stuff splits' is used without a definition or citation; please specify the exact episode split so the evaluation can be reproduced.","section":"Section 4.1"}],"recommendation":"major_revision","confidential_remarks":"The main risk is that the reported gains may be partially or entirely due to evaluation-set selection during archive construction, and even if not, the absence of error bars makes the improvement unquantified. The authors should be asked to provide the missing appendix, clarify the episode split, and re-run with multiple seeds before the paper can be considered for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou should know this paper before its numbers get quoted: it applies MAP-Elites to planning-policy templates for embodied agents, building a behavior-indexed archive of prompt styles and switching among them when the agent stalls. The combination is genuinely new—prior prompt optimization searches for a single best prompt, and prior recovery methods repair the current policy rather than switching to a behaviorally distinct one. The paper explains the offline/online loop well, and the ablations (no OS, quality-only selection) support the design. Code is released, which helps.\n\nThe soft spots are in the evaluation. Everything is point estimates: no error bars, no seeds, no significance tests. That would be acceptable for a pilot, but Table 1's headline differences (0.42→0.51, 0.24→0.43) have no uncertainty attached. Bigger issue: the paper never states that the rollouts used to build the archive are disjoint from the episodes scored in the table. Algorithm 1 rolls out every offspring to compute Q and BD; if those rollouts touch the test episodes, the reported gain is selection, not robustness. The stress-test note is right to flag this. The paper defers to an appendix that is not in the preprint, so we cannot check. The VLN-CE pilot is an 11-scene subset with a 2.1-point gain that could be noise. Hyperparameters like alpha, delta_s, and grid resolution are fixed without sensitivity analysis.\n\nNone of this sinks the idea. The mechanism is plausible, and the ablation pattern (full QD > OS + Abl. ArchSel > no OS) is consistent with the design. But the manuscript as submitted does not demonstrate the size of the claimed improvement—the missing split statement and missing variance estimates leave open the possibility that part of the gain comes from evaluating on the optimization set.\n\nWho is this for? Anyone working on LLM-agent robustness, prompt evolution, or multi-behavior control. The combination should be in the literature, and the paper deserves a serious referee—one who pushes for the appendix, a clear statement of episode separation, and uncertainty quantification. I'd take the method seriously, but not the numbers at face value.\n\nRecommendation: engage with it; send to review with requests for the missing details.","headline":"A genuinely new QD-over-prompt-templates approach with online stall switching, but the experimental reporting leaves the headline gains unverified because the evaluation split is unstated and all results are point estimates.","tokens_in":12957,"tokens_out":2326,"would_cite":false,"duration_ms":25673,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that long-horizon embodied agents fail mainly because they lock onto a single planning style, and that maintaining a behavior-indexed archive of diverse planning-policy templates—built by quality-diversity search and…","keywords":["LLM agents","prompt policy","quality-diversity","MAP-Elites","embodied planning","multimodal interaction","behavioral lock-in","policy switching"],"falsifier":"Run the full system with the behavior descriptors replaced by random values while keeping everything else fixed. If task success does not drop appreciably, the claim that behavior-aware switching (Eq. 17) is the causal mechanism for recovery is falsified. A softer test: shuffle the descriptor coordinates in the archive and check whether the chosen farthest policies still produce distinguishable rollout trajectories; if they do not, the descriptor space is not capturing what the paper says it captures.","tokens_in":11948,"feed_emoji":"🤖","tokens_out":6010,"duration_ms":52842,"temperature":0.7,"pith_summary":"The paper identifies behavioral lock-in—an agent sticking to one planning style even when that style has stopped producing progress—as a main cause of long-horizon failure in multimodal embodied agents. It proposes a two-stage remedy: offline, evolve a diverse archive of planning-policy templates (prompt scaffolds with editable rule modules) using quality-diversity search, indexing each policy by interaction intensity and goal-directedness; online, execute one policy, detect stalls as sustained lack of milestone progress, roll back to the latest checkpoint, and switch to the archive policy whose behavior is farthest from the current one. On the ThreeDWorld transport benchmark under a 2000-step cap, the full system raises food success from 0.33 (rule-based RHP) or 0.42 (CoELA with GPT-4) to 0.51, and stuff success from 0.24/0.36 to 0.43, with lower token consumption than CoELA. A pilot on VLN-CE navigation shows the same recovery mechanism improving a frozen base policy from 35.2% to 37.3% success. The sympathetic reading is that robustness in embodied planning comes from preserving several useful ways of acting and knowing when to switch among them, not from making any single policy more elaborate.","feed_headline":"Diverse policy archive lifts embodied task success from 0.33 to 0.51","feed_subtitle":"Keeping many planning styles and switching on stalls beats single-prompt planners under a strict step budget.","key_machinery":"The central object is a behavior-indexed archive of planning-policy templates, built by MAP-Elites quality-diversity search. Each policy is a template $\\tau = (r_{\\text{sense}}, r_{\\text{progress}}, r_{\\text{act}}, r_{\\text{recover}})$ whose editable rule modules dictate sensing, progress interpretation, action commitment, and recovery style. Policies are mapped into a two-dimensional behavior space via $\\mathrm{BD}(\\tau) = (b_{\\text{int}}, b_{\\text{goal}})$, where $b_{\\text{int}}$ is the fraction of information-gathering actions and $b_{\\text{goal}}$ measures how consistently the policy turns interaction into milestone-advancing progress. The archive is a $10\\times 10$ grid over this space; each cell retains the elite policy for its niche. The online mechanism that carries the argument is Eq. (17): on stall, switch to the archive policy maximizing normalized descriptor distance from the current policy, among candidates not yet tried since the last checkpoint. This farthest-policy rule is what turns offline diversity into a recovery resource.","core_discovery":"The paper argues that long-horizon failures in embodied agents often come from behavioral lock-in: a single planning style that keeps producing locally plausible but globally unproductive actions. To address this, it proposes a two-stage framework. Offline, a MAP-Elites-style quality-diversity search evolves planning-policy templates—prompt scaffolds with editable rule modules for sensing, progress interpretation, action commitment, and recovery—and organizes them into an archive indexed by two behavior descriptors: interaction intensity (the fraction of information-gathering actions) and goal-directedness (how consistently interactions become milestone progress). Each niche stores the highest-quality policy found, where quality mixes success rate and efficiency. Online, the agent executes one policy, maintains a checkpoint at each milestone, detects stalls as sustained non-advancement of the progress state (10 consecutive steps), and upon stall rolls back to the last checkpoint and switches to the archive policy whose behavior descriptor is farthest from the current one (normalized Euclidean distance in the 2D behavior space). On the TDW-MAT transport benchmark with a 2000-step cap, the full system with GPT-4 reaches 0.51 food and 0.43 stuff success, versus 0.33/0.24 for the rule-based RHP baseline and 0.42/0.36 for CoELA with GPT-4, using lower average token consumption (62.36k vs 69.42k). A pilot on VLN-CE navigation with a frozen CMA policy plus a separate recovery archive similarly improves success from 35.2% to 37.3%, suggesting the mechanism transfers beyond object transport.","pith_inferences":["A natural extension is learned behavior descriptors: instead of hand-designing interaction intensity and goal-directedness, derive descriptors from trajectory embeddings or task-progress signals so the archive generalizes to settings where the right recovery behavior is not known in advance.","The distance-maximizing switch rule treats the archive as a discrete repertoire; a continuous interpolation between policy templates could allow finer-grained behavioral control between niches, potentially improving recovery when the farthest policy overshoots the needed behavior.","The stall detector is a fixed threshold on progress-state stasis; a learned or adaptive detector that anticipates stalls from trajectory statistics could switch earlier and save steps, provided it does not cause oscillation between policies.","The pilot result suggests a modular design: a frozen base policy plus a separate recovery archive can be added to an existing agent without retraining the base, making the mechanism a plug-in robustness layer for deployed embodied agents."],"forward_implications":["If diverse policy repertoires are what drive robustness, single-prompt optimization methods such as ReAct, Reflexion, and Tree-of-Thoughts should be expected to plateau on long-horizon embodied tasks where behavioral lock-in dominates.","The same archive can serve multiple tasks without retraining: the VLN-CE pilot shows a recovery archive built from navigation rollouts transfers to a different domain (waypoint following) and improves a frozen base policy.","Because the quality-diversity variant with LLaMA-2 uses fewer tokens than CoELA with GPT-4 (51.21k) while achieving competitive or better success, diversity-based recovery need not increase inference cost.","Ablations show the archive alone is much weaker than the full system (0.37/0.25 vs 0.51/0.43 without vs with online switching), so the stall-triggered switching and rollback mechanism are what convert offline diversity into online gains."],"supporting_citations":[{"why":"Supplies the MAP-Elites archive algorithm that organizes policies into behavior-indexed niches.","marker":"[14]"},{"why":"Provides the CoELA baseline and the TDW-MAT benchmark protocol, including the RHP baseline, against which the method is compared.","marker":"[21]"},{"why":"Serves as a direct LLM-planning baseline (ReAct) in the matched 2000-step comparison.","marker":"[20]"},{"why":"Serves as the failure-recovery baseline (Reflexion) that diversity-based switching is meant to improve upon.","marker":"[16]"},{"why":"Serves as a deliberation baseline (Tree-of-Thoughts) in the matched comparison.","marker":"[19]"},{"why":"Supplies the VLN-CE benchmark and the frozen CMA policy used in the cross-domain pilot.","marker":"[12]"},{"why":"Defines the ThreeDWorld Transport Challenge environment in which the main experiments run.","marker":"[7]"}],"fun_headline_variants":["Diverse policy archive beats single-style planners on embodied tasks","Stall-triggered policy switching improves embodied agent success","Quality-Diversity search finds varied planning policies for long-horizon tasks","Archive of planning policies reduces stalls and boosts task success"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the two hand-designed behavior descriptors—interaction intensity and goal-directedness—capture the behavioral differences that actually determine whether a new policy can escape a stall; if the descriptor space is uninformative, the farthest-policy switching rule (Eq. 17) could pick a policy that is behaviorally no more useful than the stalled one.","fun_headline_variants_meta":{"raw":{"variants":["Diverse policy archive beats single-style planners on embodied tasks","Stall-triggered policy switching improves embodied agent success","Quality-Diversity search finds varied planning policies for long-horizon tasks","Archive of planning policies reduces stalls and boosts task success"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000584,"raw_usage":{"total_tokens":2844,"prompt_tokens":1138,"completion_tokens":1706,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":754,"completion_tokens_details":{"reasoning_tokens":1636}},"tokens_in":754,"tokens_out":1706,"duration_ms":11339,"temperature":1.0,"reasoning_tokens":1636,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:32:10.411475+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the full system with the behavior descriptors replaced by random values while keeping everything else fixed. If task success does not drop appreciably, the claim that behavior-aware switching (Eq. 17) is the causal mechanism for recovery is falsified. A softer test: shuffle the descriptor coordinates in the archive and check whether the chosen farthest policies still produce distinguishable rollout trajectories; if they do not, the descriptor space is not capturing what the paper says it captures.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the ThreeDWorld Transport Challenge environment in which the main experiments run."}],"review_version":1}