{"id":"a70cdacc-e68e-481e-b0d9-a05a7331ffb0","arxiv_id":"2606.31260","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"BDDL-based symbolic verification and GroupAdapt scheduling let an 8B RL planner reach 97.3 Strict-Pass on BEHAVIOR-1000 with 79 percent shorter plans than prior baselines.","lead":"The paper shows how a BDDL formal specification built from video can verify plans and supply fast rewards for training embodied planners, paired with GroupAdapt to adjust plan lengths based on group success rates. This lets an 8B model reach high success on household tasks while producing much shorter outputs than larger baselines.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"BDDL faithfulness to video evidence and physical tasks is the load-bearing assumption","rationale":"The reader's weakest_assumption directly names the single point on which every other component depends. Because the numerical headline result is produced by the verifier and reward derived from this BDDL, any undetected construction error propagates to the final scores. The structural concern survives even after the full manuscript is read; it is not resolved by additional implementation details alone.","tokens_in":1827,"tokens_out":305,"duration_ms":28296,"concrete_test":"Sample 50 BEHAVIOR-1000 tasks; have two independent annotators compare each video, the generated BDDL, and the simulator success condition; report the fraction of specs with at least one critical mismatch (missing precondition or incorrect goal predicate). If the mismatch rate exceeds 8 %, recompute the Strict-Pass numbers under corrected specs.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim rests on one automatically constructed BDDL spec serving as the shared interface for data construction, LLM verification, and symbolic reward. The video-to-BDDL parser is the sole source of this spec; any systematic mismatch (omitted predicates, incorrect object relations, or divergence from actual task success criteria) would make the verifier supply false-positive feedback and the reported 97.3 Strict-Pass / 25.9 % relative gain unreliable. The abstract states the parser exists but supplies no quantitative validation against human judgment or simulator ground truth.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that a single automatically constructed BDDL specification from a video-to-BDDL parser can serve as a shared interface for data construction, LLM-based plan verification, and symbolic reward design in embodied task planning. Combined with a GroupAdapt difficulty-aware length schedule that uses in-batch pass rate as a signal, this enables an 8B planner to reach 97.3 Strict-Pass on BEHAVIOR-1000 (25.9% relative gain over Qwen3-8B baseline, exceeding strongest large-model baseline by 3.5%), while reducing response length by 79% to 207 tokens.","tokens_in":1938,"tokens_out":383,"duration_ms":18442,"significance":"If the BDDL faithfulness assumption holds and the reported gains are reproducible, the work would demonstrate a practical route to dense, low-latency verifiable supervision for embodied planners that avoids both string matching and full simulation, enabling efficient training of smaller models with measurable gains in both accuracy and token efficiency.","major_comments":[{"comment":"Abstract: The central performance numbers (97.3 Strict-Pass, 25.9% relative improvement) rest on the video-to-BDDL parser supplying a faithful shared interface, yet the manuscript supplies no quantitative validation, error analysis, or ablation of the parser against human judgment or simulator ground truth. This is load-bearing for the claim that the verifier supplies reliable feedback.","section":"Abstract"},{"comment":"Abstract: No derivation, ablation, or sensitivity analysis is provided for the GroupAdapt schedule or its use of in-batch pass rate as the difficulty signal; the reported compression to 207 tokens and the Strict-Pass gains cannot be assessed for robustness without these controls.","section":"Abstract"}],"minor_comments":[],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for highlighting the need for validation of the BDDL parser and analysis of the GroupAdapt schedule. Both points identify gaps in the current manuscript that we will address through additional experiments and exposition in the revision.","responses":[{"response":"We agree that quantitative validation of the video-to-BDDL parser is essential to substantiate the shared-interface claim. The current manuscript relies on the parser for data construction and verification but does not report error rates or agreement metrics. In the revision we will add a dedicated section with parser accuracy against human annotations on 200 BEHAVIOR-1000 tasks and against simulator ground truth, including failure-mode categorization. This will directly support the reliability of the verifier feedback.","revision_made":"yes","referee_comment":"[Abstract] Abstract: The central performance numbers (97.3 Strict-Pass, 25.9% relative improvement) rest on the video-to-BDDL parser supplying a faithful shared interface, yet the manuscript supplies no quantitative validation, error analysis, or ablation of the parser against human judgment or simulator ground truth. This is load-bearing for the claim that the verifier supplies reliable feedback."},{"response":"We concur that the absence of derivation and controls for GroupAdapt limits assessment of robustness. The manuscript introduces the schedule but provides no ablations on the in-batch pass-rate signal or sensitivity to its hyperparameters. In the revision we will include (i) a formal derivation of the length-tolerance update rule, (ii) an ablation replacing the pass-rate signal with fixed or oracle difficulty, and (iii) sensitivity plots over batch-size and pass-rate thresholds, showing impact on both Strict-Pass and token length.","revision_made":"yes","referee_comment":"[Abstract] Abstract: No derivation, ablation, or sensitivity analysis is provided for the GroupAdapt schedule or its use of in-batch pass rate as the difficulty signal; the reported compression to 207 tokens and the Strict-Pass gains cannot be assessed for robustness without these controls."}],"tokens_in":1439,"tokens_out":435,"duration_ms":12699,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The punchline is that this approach gets an 8B planner to 97.3 strict-pass on BEHAVIOR-1000 with 79% shorter outputs by using one BDDL spec for data, verification, and rewards, plus a simple GroupAdapt scheduler that widens length tolerance for hard batches.\n\nThe new piece is the tight loop: video-to-BDDL parsing feeds an LLM verifier and a symbolic engine that runs in milliseconds, then GroupAdapt uses the batch pass rate to adjust plan lengths without extra cost. That combination is not in the baselines they cite, and it directly tackles the simulation bottleneck for household robotics.\n\nIt does the engineering cleanly enough to produce those numbers while keeping plans short. The efficiency claim looks real on its face.\n\nThe soft spot is exactly what the stress test flags. The parser is the only source of the shared spec, yet the abstract gives no human validation, no simulator cross-check, and no error rates on predicate accuracy or object relations. If the BDDL systematically misses task constraints, the verifier rewards plans that look good on paper but fail in the world, and the 25.9% lift becomes hard to trust. No ablations on the parser or on GroupAdapt are mentioned either.\n\nThis is for people building embodied planners who already work with BDDL or similar formalisms and want faster inner-loop signals. A reader who needs reproducible gains on BEHAVIOR-1000 would find the method worth trying, but they would have to add their own validation first.\n\nIt deserves peer review because the practical problem is clear and the reported efficiency is large enough to check.","headline":"The paper shows a workable way to swap slow simulation for fast BDDL-based feedback in embodied RL, but the gains rest on an unvalidated parser that could be feeding the system false signals.","tokens_in":2417,"tokens_out":413,"would_cite":false,"duration_ms":28699,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"A single BDDL specification serves as the shared interface for data construction, plan verification, and reward design in embodied task planning.","keywords":["embodied task planning","BDDL specification","symbolic verification","reinforcement learning","GroupAdapt schedule","BEHAVIOR-1000","plan verification","video-to-BDDL parser"],"falsifier":"Executing a large sample of plans that pass the BDDL verifier inside the full BEHAVIOR simulator and measuring that a substantial fraction fail to complete the task due to unrepresented physical constraints or spec inaccuracies.","tokens_in":2741,"feed_emoji":"🤖","tokens_out":810,"duration_ms":23962,"temperature":0.7,"pith_summary":"The paper establishes that embodied planners can obtain cheap, deterministic supervision by deriving one formal BDDL specification from video evidence or curated tasks. This specification feeds a video-to-BDDL parser, an LLM verifier, and a lightweight symbolic engine that returns dense feedback in milliseconds. A difficulty-aware length schedule called GroupAdapt then uses the current in-batch pass rate to give harder prompts more token budget and tightens it automatically as success improves. Together these components let an 8B model reach 97.3 Strict-Pass on BEHAVIOR-1000 while shortening outputs by 79 percent.","feed_headline":"BDDL verifier lifts 8B planner to 97.3 Strict-Pass with 79 percent shorter outputs","feed_subtitle":"Symbolic engine and group-based length schedule deliver dense rewards without full simulation on household tasks.","key_machinery":"The BDDL specification as a shared interface, together with the LLM verifier and the GroupAdapt length schedule, that supplies deterministic rewards and adaptive token budgets without full simulation.","core_discovery":"A single BDDL specification, automatically constructed from open-world video evidence or curated tasks, can serve as a shared interface for data construction, plan verification, and reward design. A video-to-BDDL parser, an LLM verifier, and a lightweight symbolic engine together supply dense feedback at millisecond latency. GroupAdapt, a difficulty-aware length schedule that uses the in-batch group pass rate as a zero-cost signal, grants hard prompts wider length tolerance that tightens as their pass rate improves. Under this guidance the 8B planner attains a Strict-Pass score of 97.3 on BEHAVIOR-1000, a 25.9 percent relative improvement over the Qwen3-8B baseline that also exceeds the stro","pith_inferences":["The same BDDL interface could support continual online refinement if the video-to-BDDL parser is run on robot camera streams.","Length compression may directly lower end-to-end latency when the planner runs on resource-limited robot hardware.","Replacing BDDL with other formalisms such as PDDL or linear temporal logic would test whether the verification-plus-adaptation pattern generalizes beyond the current specification language.","Combining the symbolic verifier with real-world execution traces could expose and correct systematic gaps between simulated and physical task requirements."],"forward_implications":["Planners receive dense deterministic rewards at millisecond latency instead of waiting for full simulation rollouts.","Response length can be compressed by nearly 80 percent while raising Strict-Pass rates above both same-size and larger baselines.","Smaller 8B models can surpass the strongest large-model baselines on household-task benchmarks.","In-batch pass rate supplies a zero-cost difficulty signal that automatically widens or tightens length tolerance during training."],"fun_headline_variants":["8B planner scores 97.3 Strict-Pass using BDDL verifier","BDDL and GroupAdapt shrink planner outputs by 79 percent","Symbolic verification boosts embodied planner to 97.3 pass rate","BEHAVIOR-1000 result 97.3 Strict-Pass from 8B symbolic RL"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"A single automatically constructed BDDL specification can faithfully serve as the shared interface for data construction, plan verification, and reward design without introducing systematic mismatches between video evidence, the formal spec, and physical task requirements.","fun_headline_variants_meta":{"raw":{"variants":["8B planner scores 97.3 Strict-Pass using BDDL verifier","BDDL and GroupAdapt shrink planner outputs by 79 percent","Symbolic verification boosts embodied planner to 97.3 pass rate","BEHAVIOR-1000 result 97.3 Strict-Pass from 8B symbolic RL"]},"model":"grok-4.3","cost_usd":0.006827,"raw_usage":{"total_tokens":3239,"prompt_tokens":801,"num_sources_used":0,"completion_tokens":79,"cost_in_usd_ticks":68274500,"prompt_tokens_details":{"text_tokens":801,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":2359,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":801,"tokens_out":79,"duration_ms":18246,"temperature":1.0,"reasoning_tokens":2359,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-01T05:18:05.049050+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Executing a large sample of plans that pass the BDDL verifier inside the full BEHAVIOR simulator and measuring that a substantial fraction fail to complete the task due to unrepresented physical constraints or spec inaccuracies.","supporting_citations":[],"review_version":1}