{"id":"c2a0809f-1bf4-4f0e-a10a-4ed3c81a79e9","arxiv_id":"2505.21981","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"BLADE learns structured, planable action representations from language-annotated demonstrations and composes them with a symbolic planner, outperforming latent and LLM/VLM baselines on new manipulation tasks.","lead":"BLADE is a system that turns a small set of language-labeled robot demonstrations into a library of high-level actions with preconditions and effects, then uses a symbolic planner to compose them for new long-horizon tasks. The result matters because it points toward robots that can generalize to new goals and disruptions without hand-coded symbolic models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The automatic predicate annotation derives all classifier labels from the same LLM-generated operators it is meant to validate; no direct check against real or simulated states is reported, so the recovered representation could be internally consistent but misgrounded.","rationale":"The most defensible reading of the central claim is that BLADE recovers preconditions and effects that are actually grounded in perception and useful for planning. That requires the LLM-generated operators to be correct, because they are the only supervision source for the predicate classifiers. I examined Section 4.2 and Algorithm 2: the label generation is indeed derived from the operators, and Algorithm 1's abstract verification only checks internal consistency, not correspondence to real states. This is the same load-bearing assumption the reader identified. I agree with the reader's conditional verdict: the concern is real but not yet demonstrated to be fatal. BLADE's Table 1 end-to-end success is genuine supporting evidence, since goal satisfaction is measured in simulator state and high success implies that at least the predicates used in those tasks are reasonably grounded. I did not elevate the predicate-name caveat from Section 3 to the primary concern because it is explicitly disclosed and Appendix A.2 provides partial evidence of automatic predicate generation; it is a scope limitation rather than a correctness risk. The proposed audit against CALVIN ground-truth states would directly test the circularity. If the audit passes, the central claim is substantially supported; if it fails, the automatic-recovery claim needs significant qualification. The absence of released code and data makes independent replication harder, but that is not by itself an argument against the central claim. Hence no verdict change beyond the reader's CONDITIONAL.","tokens_in":30221,"tokens_out":6880,"duration_ms":85792,"concrete_test":"In CALVIN split D, use the simulator's ground-truth state to audit every LLM-generated operator: for each occurrence of each behavior in the demonstration data, check whether each precondition atom was true in the actual state before execution and each effect atom true after execution, and report per-atom and per-operator accuracy. Then retrain the predicate classifiers using labels derived from the simulator ground truth instead of Algorithm 2, keeping all other components identical, and rerun the six generalization tasks from Table 1. If ground-truth-labeled classifiers substantially outperform the automatically labeled ones beyond the reported standard deviations, or if any operator has a large fraction of incorrect atoms, the automatic-recovery claim is not yet established; if the results match, the circularity concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.2's automatic predicate annotation and Algorithm 2 derive all classifier supervision from the LLM-generated preconditions and effects, not from observed states. The only filter, Section 4.1's abstract verification, checks that accumulated effects of previous LLM operators satisfy the next operator's preconditions within each demonstration trajectory; it never checks those atoms against the actual state before or after the action. Thus the pipeline is circular: the same symbolic description being 'recovered' is used to label the perceptual training data for its own groundings. If the LLM proposes an incorrect precondition or effect, the classifier learns a distorted grounding and the planner operates on an internally consistent but factually wrong model. A concrete failure mode: if an operator's effect omits a predicate that actually changes, Algorithm 2 propagates the stale label forward. For example, if place_in_drawer does not delete is-on(block, table), the is-on classifier receives positive labels on images of a block inside the drawer. CALVIN provides simulator ground-truth states, so such labels could have been audited, but the paper reports no direct verification of operator correctness or classifier accuracy against ground truth. Table 2 only compares two label-propagation schemes, both derived from the same LLM operators. End-to-end success in Table 1 is encouraging indirect evidence, but it does not isolate whether the recovered symbolic representation is correct; the real-world results are 4/6 per condition with no error bars.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces BLADE, a framework for long-horizon robotic manipulation that combines imitation learning with model-based planning. Given language-annotated demonstrations, BLADE segments trajectories into contact primitives, uses an LLM to propose PDDL-style behavior descriptions with preconditions and effects, and automatically generates observation-level labels to train visual predicate classifiers and diffusion-policy controllers. At test time, a symbolic planner composes these behaviors, with replanning after each execution. The authors evaluate in the CALVIN simulation benchmark on three generalization categories (abstract goals, geometric constraints, partial observability) and in two real-world tabletop domains, reporting substantial improvements over latent-planning and LLM/VLM baselines. The main claims are that BLADE recovers structured action representations automatically, without manual state labels or symbolic definitions, and that these representations support generalization to novel initial states, perturbations, and goals.","tokens_in":30521,"tokens_out":6917,"duration_ms":68189,"significance":"If the claims hold, BLADE would be a meaningful step toward automatic acquisition of grounded symbolic abstractions for long-horizon manipulation: it replaces manually designed predicate groundings or symbolic action definitions with LLM-generated operators combined with learned visual classifiers. The simulation results on CALVIN show clear gains over HULC, SayCan, Robot-VILA, and Text2Motion, and the ablation against Migimatsu and Bohg shows the value of the proposed label-propagation scheme over prior classifier learning. The paper also provides detailed appendices with algorithm pseudocode (Algorithms 1 and 2), full LLM prompts, and task definitions, which substantially aids reproducibility. However, as detailed below, the automatic predicate-annotation loop is not audited against ground-truth states, and the headline 'without symbolic definitions' claim is stronger than the implementation's requirement of a provided predicate list. The approach is significant if these gaps are addressed.","major_comments":[{"comment":"The supervision for every predicate classifier is generated from the LLM's own preconditions and effects, with labels assigned before and after behaviors by unrolling the symbolic operators; the same operators are then used by the planner that the classifiers feed. Section 4.1's abstract verification checks only that accumulated symbolic effects satisfy later preconditions, not that the atoms match the observed state before or after each action. If an LLM operator omits a predicate that actually changes, Algorithm 2 propagates the stale label to subsequent time steps (lines 10-24). For example, if place-in-drawer did not delete is-on(block, table), the is-on classifier would receive positive labels on images of a block inside the drawer. Because CALVIN provides simulator ground-truth states, this can be audited directly. I ask the authors to (i) evaluate the learned classifiers against ground-truth predicate values on held-out states, and (ii) report the accuracy of the LLM-proposed operators' preconditions and effects against observed state changes. Without such an audit, the F1 improvement cited in Section 5.2 and the end-to-end success rates in Table 1 are consistent with an internally self-consistent but misgrounded model, and the central claim of recovering grounded representations is not established.","section":"Section 4.2, Algorithm 2"},{"comment":"The abstract claims that BLADE recovers structured representations 'without manually labeled states or symbolic definitions,' but Section 3 states that 'BLADE requires humans to additionally provide a list of predicate names in natural language,' and the main simulation and real-world results use expert-provided predicates. Appendix A.2 shows automatic predicate generation only as an additional study (28/30 predicates match), not as part of the main pipeline. Since predicate names are themselves part of the symbolic vocabulary, the headline claim overstates what the main experiments demonstrate. Please either integrate automatic predicate generation into the main evaluation or qualify the claim to 'without manually labeled states' while stating that the predicate vocabulary is provided by the user.","section":"Abstract and Section 3"},{"comment":"The real-robot evaluation reports counts out of six trials per condition with no variance, confidence intervals, or per-trial detail, and all tasks are author-designed. With n=6, the claim that BLADE 'significantly outperforms' Robot-VILA is not statistically supportable. I recommend either increasing the number of trials, reporting exact binomial confidence intervals, or softening the significance wording. This does not undermine the simulation-based evidence, but the real-world generalization claim in the abstract should be scaled to what the data can support.","section":"Section 5.3, Fig. 5"}],"minor_comments":[{"comment":"The pseudocode uses the variable t both as the behavior index and as the time step inside GETTIMEDPREDICATES, which makes the indexing ambiguous; please clarify the notation.","section":"Algorithm 2"},{"comment":"The abstract verification threshold is not specified anywhere in the paper; please report the value used in the experiments and, if possible, a sensitivity analysis over this threshold.","section":"Algorithm 1"},{"comment":"The text reports F1 improvements of 20.7% overall, 16.3% for object states, and 38.6% for spatial relations, but Table 2 shows only planning success rates; adding a classifier-level table with per-predicate F1 would make the comparison with Migimatsu and Bohg more concrete.","section":"Section 5.2, Table 2"},{"comment":"The column headers 'State Classifier' and 'Latent Feasibility' are not explained in the caption; please state what entries such as 'N/A', 'Short', 'Long', 'Learned', and 'GT' mean.","section":"Table 1"},{"comment":"The paper does not state whether code or trained models will be released; please add a statement on code availability to improve reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The circularity concern in Section 4.2 is the main technical issue; an audit against CALVIN ground-truth states is feasible and would substantially strengthen the paper. The paper is otherwise well-executed, with strong simulation results and detailed appendices. In addition, the abstract's claim about recovering representations 'without symbolic definitions' should be aligned with Section 3's explicit requirement of a provided predicate list."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, read BLADE. The headline: a genuinely useful integration — LLM-generated PDDL operators, automatic predicate labels from those operators, learned visual classifiers, diffusion policies, and symbolic planning — that beats strong baselines on CALVIN generalization tasks and shows believable real-robot results. What's new is the auto-annotation: they don't hand-label states; they propagate labels from the LLM's preconditions and effects through the demonstration, which gives the classifiers much richer supervision than first/last frame methods. The ablation vs. Migimatsu and Bohg shows real classifier gains. The simulation numbers are solid (3 seeds, std reported), and the generalization tasks are sensible.\n\nThe stress-test concern is real and it lands. Section 4.2's Algorithm 2 labels all preconditions true before the action and effects false, then propagates until 'altered.' Those labels come from the same LLM operators the framework is supposed to recover. Abstract verification only checks symbolic consistency between accumulated effects and next preconditions — it never checks against actual simulator state, even though CALVIN provides ground truth. So if the LLM omits an effect (e.g., place_in_drawer forgets to delete is-on(block, table)), the classifier gets positive labels on images of a block inside the drawer. The F1 in Table 2 compares two label-propagation schemes built from the same operator set, so it can't catch this. End-to-end success is encouraging indirect evidence, but it doesn't isolate representation correctness. Real-world is 4/6 per condition, no error bars, author-designed tasks. Also no code or data released, which makes it hard to audit the labels.\n\nI wouldn't call this fatal — the framework works, and if the operators are mostly right the auto-annotation is a nice trick. But the central claim 'recover such structured representations automatically, without manually labeled states' needs an oracle check: compare propagated labels to simulator ground truth, or report classifier accuracy against true states on held-out CALVIN frames. That's a doable experiment and would settle it.\n\nBottom line: worth a serious referee. The paper is a solid candidate for a top robotics venue if the authors add a grounding audit, error bars on the real robot, and ideally release code/data. I'd bring it to reading group and would cite it as related work.","headline":"A genuinely useful integration of LLM-generated operators, auto-labeled predicates, and learned policies, but the automatic annotation needs an oracle check before the grounding claim is fully trusted.","tokens_in":31001,"tokens_out":1793,"would_cite":true,"duration_ms":18243,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"BLADE automatically recovers structured action knowledge from language-annotated demonstrations and plans long-horizon manipulation with it.","keywords":["long-horizon manipulation","imitation learning","model-based planning","language-conditioned policy learning","PDDL","state predicates","contact primitives","replanning"],"falsifier":"Take a trained BLADE domain and swap one effect in a single behavior description, for example make 'turn on faucet' claim is-filled(kettle) while keeping all demonstrations and training labels otherwise identical. If the resulting state classifier for is-filled still reaches high accuracy on held-out real states and plans still succeed, then the automatic recovery does not actually depend on the correctness of the LLM's proposed abstractions; if accuracy or success collapses, the paper's claims are tied to that correctness.","tokens_in":30048,"feed_emoji":"🤖","tokens_out":8839,"duration_ms":87563,"temperature":0.7,"pith_summary":"BLADE is a framework for teaching a robot long-horizon manipulation from a modest number of demonstrations that come with short language descriptions. Its claim is that, from these demonstrations alone, a robot can automatically recover a structured library of high-level actions, each with preconditions (what must already be true), effects (what becomes true), and a low-level controlling policy, without hand-labeled states or hand-written symbolic rules. The recovered library feeds a symbolic planner, so the robot can compose known behaviors to satisfy novel goals, recover from external perturbations, respect geometric constraints, and act under partial observability. If the claim holds, the bottleneck of long-horizon learning shifts from expensive manual state annotation to getting the linguistic and contact-level scaffolding right.","feed_headline":"BLADE turns language-annotated demos into reusable robot plans","feed_subtitle":"A planner chains learned skills to handle unseen states, obstacles, and hidden objects.","key_machinery":"The load-bearing object is a behavior, defined as a PDDL-style tuple $\\langle name, args, pre, eff, \\pi \\rangle$: a symbolic precondition and effect formula paired with a neural policy $\\pi$. BLADE generates the symbolic part by asking a large language model to interpret language-annotated demonstrations that have been segmented into contact primitives, and uses the generated body section to re-segment noisy trajectories. The mechanism that carries the automatic-learning claim is automatic predicate annotation: unrolling the proposed behavior sequence labels many observations as positive or negative examples for each predicate, training classifiers $f_\\theta(p): O \\to \\{T,F\\}$ without human labels. At execution time, the planner operates on this grounded abstract state, and replanning after each behavior is what turns perturbations and hidden-object situations into solvable subgoals.","core_discovery":"BLADE's central discovery is that a small fixed vocabulary of contact primitives, namely grasp, place, move, push, move-to, open, and close, gives a large language model enough structure to propose PDDL-style behavior descriptions whose preconditions and effects are causally consistent with the demonstrations. Those descriptions are then not just plan templates but also an automatic labeling mechanism: by unrolling the behavior sequence over the recorded observations, the system marks states before an action as satisfying its preconditions and states after it as satisfying its effects, generating training labels for visual state classifiers with no human annotation. With these classifiers grounding the abstract state, a planner chains the learned behaviors, and because planning is rerun after each behavior, the loop absorbs perturbations, geometric constraints, and partial observability. The paper validates this in simulation and on a real robot, reporting that the approach completes novel compositions of behaviors at higher rates than latent-planning and LLM/VLM planning baselines.","pith_inferences":["If predicate invention were fully automated, BLADE would no longer need its one remaining manual input, the list of predicate names; the appendix's predicate-generation experiment suggests this is close but not yet the paper's central claim.","The same automatically generated predicate labels could be reused as in-context examples or fine-tuning data for general-purpose vision-language classifiers, a use the paper only sketches in its appendix.","Replacing the most-likely-state planner with a belief-space or observation-based planner would be the natural upgrade for partial observability; BLADE currently gets by with optimistic visibility effects and replanning.","Because the contact-primitive segmentation relies on gripper state thresholds, the framework would need finer contact sensing to cover caging or sustained-contact manipulations, a limitation the paper states."],"forward_implications":["Any language goal expressible with the learned predicates becomes reachable without retraining, by composing existing behavior policies through the planner.","A perturbation mid-task is converted into a replanning event, so moving an object or failing an execution no longer ends the episode.","Visibility and geometric clearance, encoded as ordinary preconditions, let the planner insert search and obstacle-removal steps before the behavior that needs them.","The same demonstrations supply training signal for every predicate classifier through label propagation, removing the need for state labels or manually authored transition models.","Labeling the whole observation stream, not only segment endpoints, is what the paper's ablation identifies as the source of classifier accuracy and thus of planning success."],"supporting_citations":[{"why":"Supplies the diffusion-policy architecture used as the low-level neural controller for each learned behavior.","marker":"[1]"},{"why":"Provides the feasibility-model planning baseline that BLADE compares against both with ground-truth symbols and with learned classifiers.","marker":"[51]"},{"why":"Defines the STRIPS-style semantics of preconditions and effects that BLADE's behavior representations rely on.","marker":"[64]"},{"why":"Provides the PDDL language in which BLADE asks the LLM to write behavior descriptions.","marker":"[65]"},{"why":"Is the prior classifier-learning baseline whose first-and-last-frame labeling BLADE's automatic annotation improves on.","marker":"[66]"},{"why":"Is another prior method that trains predicate classifiers from segment endpoints, contrasted with BLADE's propagation scheme.","marker":"[67]"},{"why":"Supplies the open-vocabulary object detector used in the real-world classifier pipeline to crop relevant objects.","marker":"[68]"},{"why":"Provides the fast-forward heuristic planner used for symbolic plan generation in BLADE's bi-level loop.","marker":"[69]"},{"why":"Provides the simulation benchmark and play data used for the main evaluation.","marker":"[71]"},{"why":"Provides the hierarchical latent-planning baseline compared in the simulation experiments.","marker":"[72]"}],"fun_headline_variants":["BLADE turns demos and language into reusable robot plans","Robots learn long-horizon tasks from language-labeled demos","BLADE: LLM-crafted robot skills from demonstrations","BLADE plans by chaining learned contact primitives from demos","Language-annotated demos teach robots to plan novel actions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the LLM-proposed behavior descriptions are correct enough to be used as training labels, because the automatic predicate annotation converts those descriptions into ground-truth supervision for state classifiers without any independent check against real states.","fun_headline_variants_meta":{"raw":{"variants":["BLADE turns demos and language into reusable robot plans","Robots learn long-horizon tasks from language-labeled demos","BLADE: LLM-crafted robot skills from demonstrations","BLADE plans by chaining learned contact primitives from demos","Language-annotated demos teach robots to plan novel actions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00027,"raw_usage":{"total_tokens":1584,"prompt_tokens":861,"completion_tokens":723,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":477,"completion_tokens_details":{"reasoning_tokens":637}},"tokens_in":477,"tokens_out":723,"duration_ms":6881,"temperature":1.0,"reasoning_tokens":637,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T13:17:51.041742+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a trained BLADE domain and swap one effect in a single behavior description, for example make 'turn on faucet' claim is-filled(kettle) while keeping all demonstrations and training labels otherwise identical. If the resulting state classifier for is-filled still reaches high accuracy on held-out real states and plans still succeed, then the automatic recovery does not actually depend on the correctness of the LLM's proposed abstractions; if accuracy or success collapses, the paper's claims are tied to that correctness.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the feasibility-model planning baseline that BLADE compares against both with ground-truth symbols and with learned classifiers."},{"cited_title":"Lifschitz","cited_arxiv_id":null,"evidence_quote":"Defines the STRIPS-style semantics of preconditions and effects that BLADE's behavior representations rely on."},{"cited_title":"Aeronautiques, A","cited_arxiv_id":null,"evidence_quote":"Provides the PDDL language in which BLADE asks the LLM to write behavior descriptions."},{"cited_title":"Migimatsu and J","cited_arxiv_id":null,"evidence_quote":"Is the prior classifier-learning baseline whose first-and-last-frame labeling BLADE's automatic annotation improves on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Is another prior method that trains predicate classifiers from segment endpoints, contrasted with BLADE's propagation scheme."},{"cited_title":"Hoffmann and B","cited_arxiv_id":null,"evidence_quote":"Provides the fast-forward heuristic planner used for symbolic plan generation in BLADE's bi-level loop."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the simulation benchmark and play data used for the main evaluation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the hierarchical latent-planning baseline compared in the simulation experiments."}],"review_version":1}