{"id":"2958e579-50a9-49db-bdf2-9ad57674ff31","arxiv_id":"2607.27599","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Pairing VLM-generated action proposals with rollouts from a pose-image-conditioned video world model yields high success rates in novel simulated manipulation tasks without end-to-end policy retraining.","lead":"A robot planner pairs a vision-language model with a video world model trained on pose skeletons, imagining candidate actions and then asking the VLM to correct and rank them. In simulation benchmarks it beats end-to-end VLA policies on compositional, re-layout, and zero-shot manipulation tasks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Load-bearing assumption: forward-dynamics pose skeletons are admitted inaccurate under contact (C.1.1), yet all claimed planner gains rely on VLM ranking over these imagined rollouts; no oracle/ablation stress-tests this.","rationale":"Reader's weakest assumption and mine are the same, so agreement is 'agree'. The choice is driven by the paper's own admission (C.1.1, Fig. 6): the conditioning signal is wrong precisely in the contact-rich events that the planner is designed to optimize. The central empirical claims in Tables 3-5 are about end-to-end success on tasks composed of contacts (grasp, place, stack). The ablation table (Table 9) shows that local search and policy-rollout imagination—both driven by world-model rollouts—add 4-14 points, so the imagined rollouts are doing real work. If those rollouts are systematically inaccurate during contact, the VLM ranking can select actions that are not executable. Alternative concerns (no error bars, simulation-only, closed-source VLM) are genuine but not as structurally threatening: they are reporting/reproducibility issues, and the large reported margins make the main comparison likely qualitatively robust. The theory section, while internally coherent, is about idealized contextual MDPs with known rewards/linear structure and does not model pose-image error; it neither rescues nor refutes the empirical mechanism. Therefore the highest-value check is an oracle-conditioning ablation that directly measures how much the acknowledged pose error costs. This does not shift the reader's CONDITIONAL verdict; it sharpens what would be needed to move to full acceptance.","tokens_in":34899,"tokens_out":3570,"duration_ms":45298,"concrete_test":"Conduct an oracle-conditioning ablation on the same 50-trial protocols for PnP white mug (LIBERO-Long), PnP cream cheese/ketchup, and StackCube. Variant A: current forward-dynamics pose skeletons. Variant B: pose skeletons rendered from the simulator's true joint states (or, minimally, overlay the true gripper open/close contact state) during candidate imagination. Keep VLM prompts, search scales, and world-model training identical; report success rates and per-task 95% CIs over at least 5 seeds. If B is not substantially better than A, the contact inaccuracy is not load-bearing. If B clearly improves, the central claim must be scoped to contact-free reasoning or require contact-aware conditioning.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Central claim: WAP 'significantly outperforms' E2E policies on compositional, layout-shifted, and zero-shot tasks (abstract; Tables 3-5). The pipeline (Alg. 1) proposes actions, then optimizes and ranks candidates based on imagined world-model rollouts. To condition the world model, the paper computes future robot joint positions with forward dynamics and renders them as pose skeletons. Appendix C.1.1 explicitly says these poses are inaccurate: during contacts, reaction forces are unknown a priori, e.g. the computed skeleton shows the gripper closed while an object between the jaws keeps it open (Fig. 6). This is not a minor visualization issue: the VLM's feedback and ranking are given only the imagined videos, so systematic pose error can make failing candidates look successful (or vice versa). The fine-grained manipulation cases—mug grasping, cube stacking, precise placement—are exactly where the claimed advantage over imitation baselines is largest (Tables 4, 5, 9 with '+policy rollout imagination'). Yet the paper never tests whether faithful contact-aware pose conditioning changes outcomes. Without such a test, the empirical headline may owe more to VLM common sense and search than to the world model as physically grounded, and the acknowledged deficiency could invalidate the claimed mechanism.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes World Action Planner (WAP), a robot planning system that combines a Vision-Language Model (VLM) agent with a pose-image-conditioned video diffusion world model. The VLM proposes primitive actions, which are refined through (i) global optimization based on VLM feedback on imagined rollouts, and (ii) local grid search with VLM candidate ranking, optionally including imagined rollouts of a learned policy. The authors report large success-rate gains over end-to-end imitation baselines (π0.5, cosmos-policy) and over prior world-model-based policy-enhancement methods on LIBERO compositional tasks, LIBERO layout-shifted tasks, and Robosuite zero-shot tasks. The paper also provides theoretical results claiming model-based planning outperforms imitation learning in multi-task generalization in tabular and linear-function-approximation settings.","tokens_in":35202,"tokens_out":6962,"duration_ms":79497,"significance":"If the empirical claims hold, WAP would be a meaningful step toward test-time planning with world models and VLM reasoning, especially for compositional and layout-shifted manipulation where end-to-end policies fail. Strengths include the comparatively extensive world-model evaluation across four simulation suites and multiple embodiments, the ablations isolating global optimization, local search, and policy-rollout imagination, and a best-of-N comparison that uses ground-truth rewards as an upper bound. The theoretical appendix is detailed and builds on standard reward-free exploration and imitation-learning lower-bound techniques. However, as detailed below, several load-bearing issues remain: the physically inaccurate pose conditioning is not stress-tested, the theoretical comparison is asymmetric in information, the empirical headline lacks statistical support, and the 'zero-shot' claim is weakened by task-specific world-model fine-tuning.","major_comments":[{"comment":"The paper explicitly acknowledges that forward-dynamics pose skeletons are inaccurate under contact: the gripper is rendered closed while an object between the jaws keeps it open, and reaction forces are unknown a priori. Since the planning loop in Alg. 1 uses only the world model's imagined rollouts for VLM feedback and ranking, and since the largest gains are in contact-rich tasks (Tables 4, 5, 9), the absence of any ablation using ground-truth or contact-aware poses leaves the physical-grounding mechanism unverified. Please add an oracle experiment with simulator ground-truth joint poses (or contact-aware forward simulation) to demonstrate that the performance does not rely on systematically wrong imagined outcomes.","section":"C.1.1, Fig. 6, Alg. 1"},{"comment":"The theoretical comparison is not apples-to-apples. In the protocol, the imitation learning agent receives only expert action labels and never observes rewards, while the model-based agent collects rewards during exploration and, in the tabular theorem, is given the reward functions for all contexts. The Ω(|C|/K) lower bound for imitation learning therefore may reflect an information-structure advantage rather than an intrinsic superiority of model-based planning. Please either provide the IL agent with the same reward information, or explicitly reframe the results as a comparison between 'IL without rewards' and 'MB with rewards/known rewards.' This is important because contribution (ii) is stated as a general separation.","section":"B.2.1, Theorems 1-3"},{"comment":"Success rates are reported as single 50-trial counts with no error bars, no multiple seeds, and no statistical tests. The abstract's 'significantly outperforming' is therefore not statistically supported, especially for the smaller gaps in Table 5 (PickPlaceCan 80 vs 58). Please provide multiple seeds or confidence intervals, state the number of runs, and use a significance test where appropriate. Also, the baseline protocol varies across experiments: in §5.2.1 baselines are fine-tuned by the authors, while in §5.2.2 official checkpoints are used; details for adapting SAILOR and GPC-RANK are sparse.","section":"Tables 3-5 and §5.2"},{"comment":"The label 'zero-shot generalization' is overstated. Although no expert demonstrations are used, the world model is fine-tuned on 50 task-specific exploratory trajectories for PickPlaceCan and StackCube. This is not zero-shot with respect to the target task. Please either rename the setting (e.g., 'demonstration-free' or 'no-policy planning') or include a truly zero-shot evaluation in which the world model is not fine-tuned on the target task, to support the abstract's claim of 'zero-shot generalization scenarios.'","section":"§5.2.3 and D.2.3"}],"minor_comments":[{"comment":"Typos and spacing issues: 'a action-conditioned', 'W AMs', 'receices', 'sufficies', 'the the' in places. A careful proofread is needed.","section":"Throughout"},{"comment":"The presentation of PSNR/LPIPS pairs is dense; a separate table or clearer column separation would improve readability. It is also worth stating explicitly which metrics are higher-better versus lower-better in the caption.","section":"Table 1"},{"comment":"Some notation is introduced inconsistently (e.g., 'E[h...]' in a few displayed equations). These do not affect the logic but should be cleaned for publication.","section":"Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The paper has promising results and a substantial experimental effort, but the four major points above are load-bearing. In particular, the contact-inaccuracy concern is explicitly acknowledged in the manuscript and directly bears on the mechanism claimed in the abstract. The theoretical comparison should be reframed as an information-structure separation rather than a general model-based-vs-imitation result. I believe these issues are addressable with additional experiments and reanalysis, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, here's my take. The paper deserves a serious look, not a desk reject. Its core contribution is real: they condition a video-diffusion world model on pose skeletons computed from forward dynamics of the action, rather than extracting poses from future frames, and they wrap it in a VLM propose-optimize-search planner that operates over imagined rollouts. That combination is new, and the empirical results are striking — near-zero success for VLA/WAM baselines on the compositional and layout-shifted tasks versus 66–90% for their planner. The ablations show each component (global optimization, local search, policy-rollout imagination) adds value, which is well done. The theoretical section is coherent but idealized; it's a bonus, not the main load.\n\nThe soft spot the stress-test flags is real and should be front and center in review: Appendix C.1.1 openly admits the forward-dynamics pose skeletons are inaccurate under contact (gripper shown closed while an object holds it open). The planner's VLM ranking and feedback are based on videos imagined by a world model conditioned on these skeletons. If the skeleton systematically misrepresents contact, the imagined rollouts could be misleading, and the whole 'imagination-based refinement' could be selecting actions for the wrong reason. The paper does not ablate this — no oracle conditioning, no contact-aware skeleton, no test of whether the VLM's rankings change when the pose images are ground-truth. That's a genuine hole, but not necessarily fatal: the world model is trained under the same pose-image distribution, so it may have learned to compensate. The paper's own knock-over qualitative results suggest the model can produce physically realistic outcomes despite the skeleton. Still, the authors need to show this rather than assert it.\n\nOther issues are the usual ones: success rates are single 50-trial counts without error bars, some baselines are evaluated with official checkpoints rather than retrained under the exact protocol, evaluation is simulation-only, and there's no code or data. None of these are disqualifying for a first submission, but they cap the confidence level.\n\nBottom line: this is a well-engineered, honest paper on an important problem. It belongs in peer review. The main thing I'd require is a robustness check on the pose-conditioning under contact — either an oracle ablation or a comparison with a contact-aware conditioning — plus variance reporting for the main tables. Who should read it: anyone working on robot world models, VLM-based planning, or cross-embodiment generalization. I'd cite it if I were in that space.","headline":"Genuinely new integration of VLM reasoning with a pose-image world model; strong simulation results, but the known-inaccurate pose conditioning under contact is never stress-tested.","tokens_in":35659,"tokens_out":3170,"would_cite":true,"duration_ms":38295,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"World Action Planner claims that letting a vision-language model propose and refine actions inside an action-conditioned world model beats end-to-end imitation policies on compositional, novel-layout, and zero-shot manipulation tasks.","keywords":["robot planning","world models","vision-language models","generalization","imitation learning","model-based planning","action conditioning","manipulation"],"falsifier":"Run the planner on a set of contact-rich manipulation tasks (inserting, pressing, stacking with tight tolerances) while logging both the world model's imagined future frames and the true environment rollouts. If the VLM's chosen candidate correlates no better than chance with ground-truth success, or if replacing the learned world model with ground-truth simulator rollouts dramatically improves success while the learned model does not, then the claim's load-bearing premise—faithful imagination—is refuted.","tokens_in":34773,"feed_emoji":"🤖","tokens_out":7722,"duration_ms":73380,"temperature":0.7,"pith_summary":"This paper tries to establish that the bottleneck for robot generalization is not data or policy architecture but the absence of test-time reasoning about physical consequences. It introduces World Action Planner, which lets a vision-language model propose primitive actions, then imagines those actions in a learned, action-conditioned world model and iteratively optimizes and searches over candidate actions using feedback and ranking. The central claim, backed by experiments on long-horizon composed tasks, shifted object layouts, and zero-shot manipulation, is that this planning loop massively outperforms end-to-end imitation policies, which often fail entirely once the test scene or task departs from training. The paper also gives theoretical arguments that model-based planning shares dynamics data across tasks and therefore has a sample-complexity advantage over imitation learning in multi-task settings.","feed_headline":"Plan by imagining: robot planner generalizes where imitation fails","feed_subtitle":"Planning in a learned world model succeeds on 66-90% of shifted layouts and 72-78% of composed tasks.","key_machinery":"The central object is the pose-image conditioned world model: a multi-view video diffusion model that takes as conditioning clean pose skeleton images of the robot, rendered by executing the proposed actions through forward dynamics, and predicts future video frames. The pose skeletons are computed from the robot's own kinematics, not learned from pixels, so the model can imagine the physical consequences of arbitrary, even out-of-distribution, actions. This imagined rollout is then fed back to a VLM agent in three ways: to propose initial primitive moves, to provide high-level corrective feedback on safety and clearance via 'global optimization', and to rank a grid of candidate actions via","core_discovery":"The paper claims that a robot can solve novel manipulation problems by reasoning over imagined futures rather than regurgitating training motions. Its core discovery is that an action-conditioned world model, conditioned not on low-dimensional action vectors but on pose skeleton images computed from forward dynamics of the proposed actions, produces rollouts that a vision-language model can inspect, critique, and rank. On four compositional long-horizon tasks composed from seen primitives, the planner achieves 72-78% success where end-to-end baselines score near zero; on six tasks with shifted object layouts it achieves 66-90% using only five demonstrations per task; and on two zero-shot man","pith_inferences":["The 'policies as tools' framing suggests a modular robot-learning stack where local policies handle in-distribution atomic skills and the VLM-plus-world-model planner orchestrates them; this could combine the robustness of specialized policies with the flexibility of planning.","The pose-image conditioning trick is transferable: any controlled generative video model could be steered by rendering the agent's articulated state as a skeleton or keypoint overlay, without retraining the conditioning mechanism.","A natural stress test follows from the paper's own caveat: tasks with heavy contact forces (deformable objects, tight fits, regrasping under load) are where the forward-dynamics pose approximation should break down most; measuring planning success across a contact-richness axis would reveal the boundary of the claim.","The theoretical separation suggests that scaling the VLM's ability to judge task success (as a reward/evaluator model) may be a more efficient investment than scaling demonstration data or policy networks."],"forward_implications":["End-to-end policies that memorize training motions will continue to fail when scenes or task compositions change; a planning loop that reasons over imagined consequences is a viable alternative for out-of-distribution manipulation.","The VLM does not need to output precise coordinates; it only needs to rank candidate outcomes, a capability that combines with the world model to provide metric grounding.","Policy learning becomes far more sample-efficient: with a good world model and VLM planner, a handful of demonstrations per task suffices, because the policy only handles local, in-distribution atomic skills.","Model-based exploration across tasks amortizes data collection: a shared dynamics model makes each new task's reward function cheap to learn, whereas imitation learning must re-learn a distinct policy per task."],"fun_headline_variants":["World-model planning beats imitation without new training","Imagine and plan: robot generalizes to new scenes by reasoning","Why imitate when you can imagine? Robot planner uses world model","Planning over imagined futures: robot solves novel tasks zero-shot","From five demos to 90% success: robot plans by imagining"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole pipeline depends on the world model's imagined rollouts being faithful enough to real physics—especially contact events—that a VLM's feedback and ranking over imagined futures selects actions that also succeed in the real environment; the paper itself notes that forward-dynamics-computed pose images misrepresent contact (e.g., a gripper shown as closed while actually holding an object), which could make imagined outcomes misleading exactly in the fine-grained graspi","fun_headline_variants_meta":{"raw":{"variants":["World-model planning beats imitation without new training","Imagine and plan: robot generalizes to new scenes by reasoning","Why imitate when you can imagine? Robot planner uses world model","Planning over imagined futures: robot solves novel tasks zero-shot","From five demos to 90% success: robot plans by imagining"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000165,"raw_usage":{"total_tokens":1040,"prompt_tokens":653,"completion_tokens":387,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":397,"completion_tokens_details":{"reasoning_tokens":304}},"tokens_in":397,"tokens_out":387,"duration_ms":4803,"temperature":1.0,"reasoning_tokens":304,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T04:54:10.376709+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the planner on a set of contact-rich manipulation tasks (inserting, pressing, stacking with tight tolerances) while logging both the world model's imagined future frames and the true environment rollouts. If the VLM's chosen candidate correlates no better than chance with ground-truth success, or if replacing the learned world model with ground-truth simulator rollouts dramatically improves success while the learned model does not, then the claim's load-bearing premise—faithful imagination—is refuted.","supporting_citations":[],"review_version":1}