{"id":"145d3d4d-5f1a-40ec-8258-15e6dcfc944d","arxiv_id":"2505.19933","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A new PDDL benchmark and modular test suite shows that LLMs reject clear safety violations but fail to plan around subtle physical hazards.","lead":"This paper introduces EMBODYGUARD, a benchmark of 942 PDDL household scenarios, and SAFEL, a framework that tests whether LLMs refuse unsafe commands and can plan safely through three separate reasoning stages. Across 13 LLMs, models usually reject overtly harmful commands but frequently miss subtle situational hazards, with the best model completing safe plans less than half the time.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Appendix N.4 Action Sequencing prompt restricts outputs to primitive actions, but Sit ground-truth plans require scenario-specific actions like unplug_wire; the headline 44.7% success rate may be an interface artifact.","rationale":"The reader's weakest assumption was that the GPT-4o/human PDDL axioms correctly encode physical safety. That is a real concern, but it applies to the benchmark's construct validity broadly and is partly mitigated by two-stage human review. The stronger, more checkable threat is the mismatch in Appendix N.4: the action sequencing task as specified cannot express the very safety actions the Sit scenarios require. This directly threatens the headline quantitative claim (o1 SR=44.7%) rather than only the benchmark's interpretation. If the actual experimental interface differs from Appendix N.4, the paper must say so and release exact prompts; if it does not differ, the low success rates and missing-step dominance are substantially explained by an underspecified action space. I do not move the verdict from conditional: the qualitative finding that models omit safety-relevant steps may survive a corrected run, and the appropriate remedy is to rerun with the full action vocabulary and release artifacts. I therefore keep the reader's conditional verdict and add this specific condition.","tokens_in":26432,"tokens_out":9262,"duration_ms":104611,"concrete_test":"Re-run Sit Action Sequencing on the same scenarios for o1 and GPT-4o with two interfaces: (a) the Appendix N.4 prompt verbatim, and (b) the same prompt augmented with the scenario's new_actions_description and its action names added to the allowed set (or with the full PDDL domain provided). Compare success rate and missing-step error rate. If SR rises materially (e.g., >10 points) or missing-step ER falls substantially, the headline 44.7% figure is partly an interface artifact and the paper should re-report with the corrected action space.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 and Table 4 present o1's 44.7% action-sequencing success rate and 34.0% missing-step error as evidence that LLMs fail at safe planning. However, the Action Sequencing prompt in Appendix N.4 defines an \"Available Actions\" list containing only primitive actions (NAVIGATE_TO, GRASP, PLACE_ONTOP, PLACE_INSIDE, RELEASE, OPEN, CLOSE, COOK, FREEZE, UNFREEZE, SLICE, SOAK, DRY, TOGGLE_ON, TOGGLE_OFF, PLACE_NEXTTO, PLACE_UNDER, PLACE_ONFLOOR) and explicitly states: \"Action name must be from the above list.\" The Sit scenarios in Appendix K and Section 2.2 introduce scenario-specific hazard-mitigating actions such as unplug_wire and fill_kettle_with_water, which the ground-truth safe plans require. If this prompt is what produced Table 4, then for every scenario whose only safe plan uses a new action, no legal output can reach the safe goal: the model must either violate the constraint (counted as grammar/affordance error) or emit a primitive-only plan that cannot remove the hazard (counted as missing step/unmet goal). The reported dominant failure would then be an artifact of the prompt's action vocabulary rather than a model-level planning deficiency. Because the abstract and introduction use this exact number to support the central claim, this is the load-bearing threat.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper introduces SAFEL, a framework for evaluating the physical safety of LLMs in embodied decision making, and EMBODYGUARD, a PDDL-based benchmark of 942 LLM-generated scenarios (541 malicious, 402 situational). SAFEL decomposes plan safety into four modules: Command Refusal, Goal Interpretation, Transition Modeling, and Action Sequencing, and reports results for 13 open- and closed-source LLMs. The headline finding is that models are generally good at refusing overtly dangerous commands but fail at safe planning, with the best model, o1, achieving only 44.7% success on Action Sequencing and Missing Step errors dominating the failures.","tokens_in":26787,"tokens_out":10425,"duration_ms":100462,"significance":"The modular PDDL-based decomposition and the hazard taxonomy are valuable contributions; if the evaluation is corrected, the benchmark could support fine-grained safety diagnosis for embodied LLM agents. The authors provide detailed prompts and appendices, use human filtering, and evaluate a broad model zoo, which are strengths. However, the central quantitative claim is currently not well supported: the Action Sequencing evaluation appears to systematically disadvantage models by restricting the action vocabulary to primitives, and other validation gaps further weaken the results. The framework is promising, but the experimental evidence requires substantial revision before the main conclusions can be accepted.","major_comments":[{"comment":"The Action Sequencing prompt in Appendix N.4 defines an 'Available Actions' list containing only primitive actions and states that the action name must be from this list. The ground-truth plans for EMBODYGUARD sit scenarios, however, include scenario-specific new actions such as unplug_wire and fill_kettle_with_water (Appendix K). Under this prompt, models cannot legally output the hazard-mitigating actions that the ground-truth safe plans require; they either violate the stated constraint (incurring grammar/affordance errors) or omit the required step, which is then counted as a Missing Step error. Consequently, the Missing Step rates in Table 4 (e.g., 34.00% for o1) and the 44.75% Action Sequencing success rate in Table 3 are confounded by the prompt's restricted vocabulary and do not validly measure planning ability. This directly undermines the claim in the abstract and Section 6 that LLMs struggle with safe action sequencing. The authors must revise the Action Sequencing interface to expose the full set of domain actions (including new actions) or restrict the evaluation to scenarios whose safe plans use only primitive actions.","section":"Section 3.2, Table 4, Appendix N.4"},{"comment":"The scenario-generation prompts require that each scenario have exactly one unique valid plan (Appendix L.3), but the symbolic verifier described in Section 2.3 checks only whether a valid plan exists; uniqueness is not verified. Because the Action Sequencing and Goal Interpretation evaluations compare against a single ground-truth plan, a scenario that admits additional safe plans can cause a model to be incorrectly penalized for producing a valid alternative. The paper should report a uniqueness check (e.g., bounded plan enumeration) or change the success criterion to accept any plan that achieves the goal without violating safety constraints.","section":"Section 2.3, Appendix H, Appendix L.3"},{"comment":"The ground-truth scenarios, PDDL problems, and plans are generated by GPT-4o (one of the evaluated models) and then reviewed by two human annotators. Since GPT-4o is itself evaluated on this benchmark, its high scores on Goal Interpretation and Transition Modeling may partly reflect self-agreement with the data-generation process, while other models are at a disadvantage. The paper provides no analysis of inter-annotator agreement or of the sensitivity of results to the generator model. At minimum, the authors should report the human-annotation agreement and ideally evaluate on a human-authored subset to quantify this circularity.","section":"Section 2.2–2.4, Table 1"},{"comment":"The Command Refusal Test prompt explicitly provides the classification categories and examples, including 'Safe, o2 (Safe Command but Potentially Risky Due to Initial Conditions)' with instructions to accept such commands. This label leakage is the likely reason that all 13 models achieve a perfect 100.0% acceptance recall on the Sit subset in Table 3, a result that is implausible without strong prompting hints. The refusal test should be administered without exposing the expected answer categories, for example by asking for a free-form safety judgment and then mapping the response to accept/refuse.","section":"Appendix N.1, Section 3.1, Table 3"}],"minor_comments":[{"comment":"The abstract states 942 scenarios, but the numbers 541 + 402 in Section 2.2 sum to 943. Please reconcile and ensure all counts are consistent.","section":"Section 2.2"},{"comment":"'This stages allow' should be 'These stages allow'.","section":"Section 1"},{"comment":"The planner is called 'Fast Forward (Helmert, 2006)' in Section 2.1 and 'Fast Downward (Helmert, 2006)' in Section 2.3; these are different planners with different authors, and the reference is to Fast Downward. Please correct the attribution.","section":"Section 2.1 and Section 2.3"},{"comment":"The example output uses actions such as 'CLEAN' and 'SOAK' where 'CLEAN' is not in the listed Available Actions. This internal inconsistency makes it difficult to determine the actual allowed action set and should be fixed.","section":"Appendix N.4"},{"comment":"The paper states that scenarios requiring new actions were not simulated in iGibson. Please clarify that the Action Sequencing module uses symbolic execution rather than the iGibson simulator, and temper the claim of 'simulation-based execution' in Section 1 accordingly.","section":"Appendix E"},{"comment":"Claims such as 'the reasoning models outperform the others on this module' are made without significance testing or confidence intervals; given the small numbers (n~400 scenarios spread across many error categories), please report variance or statistical tests.","section":"Section 4.2"}],"recommendation":"major_revision","confidential_remarks":"The framework and benchmark have real potential, but the Action Sequencing prompt mismatch is a serious flaw that invalidates the headline success rates. The authors should re-run the experiments with a corrected prompt that includes the scenario-specific actions, and also address the uniqueness and circularity issues. I would be willing to review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, this is a genuinely useful benchmark contribution: EMBODYGUARD gives you 942 PDDL scenarios spanning explicit malicious and implicit situational hazards, and SAFEL decomposes planning into refusal, goal interpretation, transition modeling, and action sequencing, which is the right way to localize where safety reasoning breaks down. The central qualitative finding—that current LLMs are good at refusing overtly dangerous instructions but much weaker at anticipating subtle risks in planning, especially transition modeling—is believable and consistent with other recent results.\n\nSecond, you should not trust the headline action-sequencing numbers until the authors clarify the prompt. The stress-test note is onto something. The Action Sequencing prompt in Appendix N.4 lists only primitive actions and says 'Action name must be from the above list,' but the Sit ground-truth plans in Appendix K require scenario-specific actions like unplug_wire and fill_kettle_with_water. If that prompt is what generated Table 4, then for every scenario whose only safe plan uses a new action, the model physically cannot succeed: it must either violate the constraint (grammar error) or emit a primitive-only plan that leaves the hazard in place (missing step or unmet goal). That would make the 44.7% success rate and the dominance of missing-step errors an interface artifact. The appendix is internally inconsistent—the worked example uses CLEAN, which isn't in the list—so it's possible the real prompt was adapted to include new actions. The paper never says so explicitly. This needs to be fixed or explained before the specific numbers can be used.\n\nOther soft spots are real but less severe. The ground truth is generated by GPT-4o and then GPT-4o is evaluated on it; the self-agreement bias doesn't kill the comparative claim across 13 models, but it does mean all models are scored against one model's notion of physical safety. The Command Refusal prompt hands the model the answer taxonomy with examples, so the near-perfect refusal recall is not a challenging measurement. And there is no release link for the benchmark.\n\nOn balance, the benchmark is worth having and the modular diagnostic is a step forward. The paper deserves peer review, not desk rejection, but the authors need to re-run action sequencing with an action vocabulary that includes the newly defined actions, and release the data. If that resolves the mismatch, I'd be happy to cite it. As is, treat the qualitative conclusion as likely robust and the exact error rates as unverified.","headline":"Useful benchmark and a plausible core finding, but the headline action-sequencing numbers may be partly an artifact of a prompt that forbids the very actions the ground truth requires.","tokens_in":27265,"tokens_out":4018,"would_cite":false,"duration_ms":41477,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that even the best LLMs are unsafe as embodied decision makers: they refuse overtly harmful commands reliably, yet the strongest model completes a fully safe action plan only 44.7% of the time, usually because a required…","keywords":["LLM safety evaluation","embodied decision making","physical safety","PDDL planning","action sequencing","command refusal","household robotics"],"falsifier":"Execute the benchmark's situational scenarios in a full physics-based simulator, including the scenarios with newly defined safety actions that the paper's pilot simulation deliberately excluded (Appendix E), and compare actual outcomes with the benchmark's error labels; if a plan the symbolic executor flags as a missing-step failure turns out physically harmless when executed, or a \"ground-truth safe\" plan causes harm, EMBODYGUARD would be measuring its own axioms rather than physical safety.","tokens_in":26254,"feed_emoji":"🤖","tokens_out":13843,"duration_ms":131657,"temperature":0.7,"pith_summary":"This paper argues that LLM-based embodied agents are not yet safe for physical deployment, even when they reliably refuse overtly harmful commands. It introduces SAFEL, an evaluation framework that splits embodied safety into command refusal and plan safety, and EMBODYGUARD, a benchmark of 942 household scenarios spanning deceptively ordinary instructions with hidden physical hazards alongside overtly malicious ones, all written in the Planning Domain Definition Language (PDDL) so that every plan can be checked against formal rules. Across 13 state-of-the-art LLMs, the authors find that most models refuse explicit harmful commands at 82.8–99.1% recall, yet the best model, o1, completes a fully safe action plan only 44.7% of the time. The dominant failure mode is the missing step: models omit the safety-critical action, such as unplugging a live wire before boiling a kettle, that keeps an otherwise normal task from causing harm. If the results hold, refusal-style safety tests give false confidence, and embodied safety claims must be verified by plan execution in a simulated environment rather than by what the model says it would refuse.","feed_headline":"Best LLM completes only 44.7% of safe robot plans","feed_subtitle":"Models refuse obvious harm, yet omit a necessary safety step in over half of everyday household tasks.","key_machinery":"The carrying mechanism is the PDDL plan: every EMBODYGUARD scenario is written in the Planning Domain Definition Language, whose actions carry explicit preconditions and effects, so an LLM's plan can be replayed against a shared, hand-written iGibson household domain by a symbolic executor. The executor checks each action's name, argument types, and preconditions against the evolving formal state and classifies every failure into one of six categories: Missing Step, Affordance error, Wrong Temporal order, Additional Step, Unmet Goal, or Grammar error — a taxonomy that turns \"the plan failed\" into a specific, localizable diagnosis. The framework then attributes each failure to one of three reasoning modules — Goal Interpretation, Transition Modeling, and Action Sequencing — plus the Command Refusal test, so a model's weakness is pinned to a single competence rather than to perception, grounding, or motor control. Newly defined actions such as unwrap_foil and unplug_wire carry the scenario-specific hazards, which is what lets the benchmark test subtle risks while staying inside the formal substrate.","core_discovery":"SAFEL and EMBODYGUARD answer one question: where does safe embodied decision-making break down inside an LLM? The paper's answer is that the breakage sits not in refusal but in planning. Models refuse overtly malicious commands at high recall and accept benign ones perfectly, but they fail to reason about the intermediate conditions that make an action safe: they mispredict an action's effects and preconditions, and they omit or misorder the steps that neutralize hazards. On the situational subset, all 13 evaluated models failed to produce a fully safe, executable plan in a majority of scenarios; the best model, o1, succeeded only 44.7% of the time, with missing-step errors the largest single cause of failure across all strong models. The paper concludes that current LLMs lack a reliable working model of state transitions and action preconditions, and that plans must be checked by simulated execution rather than by surface inspection.","pith_inferences":["A direct next step the paper does not take is to use the symbolic executor's missing-step signal as a training or fine-tuning signal, turning EMBODYGUARD from a diagnostic into a data source for repairing the very failure it measures.","Because all scenarios share one hand-written iGibson domain, the observed failure pattern could be partly an artifact of that rule book; porting the benchmark to a second, independently authored domain would test whether missing-step errors are a general property of LLM planning or specific to this axiomatization.","A middle-ground capability the current design leaves invisible is conditional acceptance — accepting a command only together with the neutralizing safety step — which sits between the binary refusal test and the full plan test and could reveal competence the two-stage design does not score."],"forward_implications":["Refusal accuracy cannot certify embodied safety: most models refuse overtly malicious commands at 82.8–99.1% recall, yet every evaluated model failed to produce a safe executable plan in a majority of the situational scenarios.","The dominant failure is missing steps, not wrong actions: across the five strongest models, omitting a required hazard-neutralizing action accounts for roughly 29–42 percentage points of the overall error rate, so safety work should target precondition verification and transition modeling.","Text-level review of a plan is insufficient: the paper's symbolic executor reveals large failure rates in plans that look reasonable on the surface, so runtime simulation must be part of any embodied-safety claim.","Reasoning-augmented inference does not by itself deliver physical safety: R1-distilled models trail standard models such as GPT-4o and Llama-3.3-70B on several transition-modeling metrics.","The modular decomposition isolates failure stages, so interventions (better goal parsing, transition training, sequencing constraints) can be evaluated independently rather than through a single end-to-end success rate."],"supporting_citations":[{"why":"Supplies the modular, simulator-agnostic evaluation design (Embodied Agent Interface) that SAFEL extends by decomposing planning into Goal Interpretation, Transition Modeling, and Action Sequencing.","marker":"Li et al. 2024a"},{"why":"Defines PDDL, the planning formalism in which every EMBODYGUARD scenario and SAFEL check is written; the precondition/effect structure is what makes safety machine-checkable.","marker":"Aeronautiques et al. 1998"},{"why":"Provides the Fast Downward planner used to verify scenario executability and to re-validate corrected PDDL during benchmark construction.","marker":"Helmert 2006"},{"why":"Supplies the iGibson household simulation environment whose object types, predicates, and primitive actions define the shared domain all scenarios use.","marker":"Li et al. 2021"},{"why":"Provides the BEHAVIOR benchmark's 100 household activities and objects as seed material for generating EMBODYGUARD_mal and EMBODYGUARD_sit scenarios.","marker":"Srivastava et al. 2022"},{"why":"GPT-4o generated the initial pool of candidate PDDL scenarios that, after symbolic verification and human review, became EMBODYGUARD's ground truth.","marker":"OpenAI 2023"}],"fun_headline_variants":["Top LLM fails safe planning in 55% of embodied tasks","LLMs refuse harmful commands but miss hidden risks in plans","Best robot-brain model only 44.7% safe on everyday tasks","Safety gap: LLMs ace refusal tests, fail plan checks","Why LLMs fail safety: missing steps in plans, not refusal"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole evaluation rests on the assumption that the hand-written rule book used to judge plans — the Planning Domain Definition Language (PDDL) axioms specifying each household action's preconditions and effects — correctly matches how physical harm actually happens, so that a plan the rules approve would really be safe and a plan the rules flag would really cause harm; this is the paper's declared \"we assume that the domain rules are predefined\" (Section 2.1), and Section 7 concedes that automated verification alone cannot guarantee the rules preserve their intended commonsense meaning.","fun_headline_variants_meta":{"raw":{"variants":["Top LLM fails safe planning in 55% of embodied tasks","LLMs refuse harmful commands but miss hidden risks in plans","Best robot-brain model only 44.7% safe on everyday tasks","Safety gap: LLMs ace refusal tests, fail plan checks","Why LLMs fail safety: missing steps in plans, not refusal"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000582,"raw_usage":{"total_tokens":2740,"prompt_tokens":950,"completion_tokens":1790,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":566,"completion_tokens_details":{"reasoning_tokens":1715}},"tokens_in":566,"tokens_out":1790,"duration_ms":13242,"temperature":1.0,"reasoning_tokens":1715,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:03:42.267035+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Execute the benchmark's situational scenarios in a full physics-based simulator, including the scenarios with newly defined safety actions that the paper's pilot simulation deliberately excluded (Appendix E), and compare actual outcomes with the benchmark's error labels; if a plan the symbolic executor flags as a missing-step failure turns out physically harmless when executed, or a \"ground-truth safe\" plan causes harm, EMBODYGUARD would be measuring its own axioms rather than physical safety.","supporting_citations":[],"review_version":1}