{"id":"7fa150de-0418-4ed7-8249-e508a23cc741","arxiv_id":"2509.03383","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"The authors build a safety-centered benchmark and attack method that induces vision-language-action robot policies to violate ISO-based safety rules in a majority of tested episodes.","lead":"This paper introduces a safety benchmark and an adversarial attack framework for vision-language-action robots, classifying violations as critical, dangerous, or risky using ISO human-robot safety standards. It reports that the attack can push robot policies to violate these rules in over half of tested simulation episodes, with a smaller physical robot demonstration.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unspecified safety thresholds in Eqs. 5-7 define the ASR metric; without numeric values and a stated aggregation rule, the '>50% across all safety categories' headline is unverifiable, and Table 2's reported rates do not obviously match the abstract.","rationale":"I read the paper as claiming: (i) a principled ISO-grounded safety definition, (ii) a benchmark, and (iii) an attack that yields >50% ASR across categories, validated by physical replay. The attack-optimization pipeline (leader + PGD) is plausible and code is promised, so I do not see an internal inconsistency in the method itself. The weakest link is the definition of the outcome variable. Eqs. 5-7 are inequalities with unnamed constants; ASR is literally a count of violations of those inequalities. Without the constants, the paper cannot distinguish 'attack succeeded' from 'threshold was conveniently placed.' This is not a disagreement with consensus; it is a missing-parameter problem that the authors can fix in one table. The mismatch among abstract, introduction, and Table 2 suggests the aggregation rule is also under-specified. I therefore agree with the reader's weakest_assumption. A secondary weakness is the open-loop replay in Sec. 6.3, but I do not make it primary because the simulated ASR is the central claim and the threshold issue must be settled first. The verdict stays conditional: if the released values reproduce Table 2, the paper's core evidence stands; if not, the headline claim should be revised.","tokens_in":21623,"tokens_out":5753,"duration_ms":60879,"concrete_test":"Extract from the released repository (github.com/RLCLab/Annie) the exact numeric values of Tcritical, T_ee_dangerous, T_env_dangerous, and the forbidden-object lists used to label each attack/benign episode. Using those values, rerun the 20 test episodes for the 'Cut the apple with knife' row (Baku and ACT) in ManiSkill and recompute ASR as episodes whose trajectories violate Eqs. 5-7. If the repository does not contain these constants, re-derive them from ISO/TS 15066 for the Franka Panda with the measured commanded end-effector speeds, rerun the same episodes, and compare. If the recomputed ASR falls below 0.5 for a category, or differs materially from Table 2's 0.7/0.5 entries, the '>50%' central claim is not supported as stated.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central quantitative claim is ASR exceeding 50% in all safety categories. That metric is defined in Sec. 4.2 as the percentage of instances where safety constraint rules are violated, and the rules are Eqs. 5-7. None of the quantities in those inequalities is given a numeric value: Tcritical (Eq. 5), T_ee_dangerous and T_env_dangerous (Eq. 6), and the forbidden-object sets in Eq. 7 are never specified in the text, the experimental setup, or the appendix. There is no mapping from ISO/TS 15066 recommended separation distances or speed limits to these thresholds, so a reader cannot tell whether an ASR of 0.7 reflects a genuinely unsafe trajectory or a threshold placed inside the normal reach envelope. This is load-bearing because every headline number, the attack-leader ablation, and the sparse-attack comparison inherit this definition. The problem is compounded by aggregation inconsistency: the abstract states >50% across all categories, the introduction reports 52%/67%/50% for critical/dangerous/risky, and simple row averages from Table 2 give roughly 72% critical, 52% dangerous, and 50% risky for the two models combined, with one cell as low as 0.1. The exact filtering and aggregation rule is not stated. A secondary concern is that the physical robot validation (Sec. 6.3) replays attacked vision-action sequences open-loop rather than running the attack live through the policy in a closed loop; while this demonstrates that the generated actions are physically executable, it does not demonstrate online sensor-driven attack execution. The threshold issue is primary.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a safety taxonomy for embodied AI systems—critical, dangerous, and risky—grounded in ISO/TS 15066 and ISO 13855, and contributes ANNIE-Bench, a simulation benchmark of nine scenarios with 2,400 video-action sequences, and ANNIE-Attack, a white-box framework that uses an attack leader model to decompose long-horizon goals into frame-level targets before applying PGD. The attack is evaluated on ACT and Baku policies, with reported attack success rates (ASR) claimed to exceed 50% across safety categories, plus sparse and adaptive variants and a physical UR3 replay in which a knife is induced to approach a human in 4 of 10 trials. The paper also defines auxiliary metrics (Action Consistency, Action Deviation, Task Success Rate Change) and argues that current accuracy-based metrics are insufficient for embodied safety.","tokens_in":22082,"tokens_out":6864,"duration_ms":67383,"significance":"If the central claims hold, this is a useful step toward safety-centric evaluation of vision-language-action policies. The ISO-inspired taxonomy is a reasonable organizing principle, and the attack-leader decomposition is a sensible way to make long-horizon adversarial attacks tractable. The paper also ships code and a benchmark, reports an ablation of the attack leader, includes held-out test episodes, and attempts a physical validation; these are concrete strengths. However, the quantitative conclusions are currently load-bearing on unspecified safety thresholds and inconsistent aggregation rules, and the physical experiment is an open-loop replay rather than a closed-loop demonstration. These issues must be fixed before the headline claims can be accepted at face value.","major_comments":[{"comment":"The ASR metric and the attack-leader training labels are defined by the inequalities in Eqs. (5)-(7), yet no numeric value is given for Tcritical, T_ee_dangerous, T_env_dangerous, or the forbidden-object sets in Eq. (7). There is also no mapping from ISO/TS 15066/ISO 13855 to these thresholds, and no statement about whether they were fixed a priori or calibrated after observing policy behavior. Since every headline ASR in Table 2 and the abstract inherits these constants, the quantitative claims are unverifiable as written. Please add explicit threshold values, their derivation from the standards, the episode-level counting rule (e.g., any violating frame vs. majority), and the aggregation rule across scenarios and models.","section":"Sec. 3.1, Eqs. (5)-(7)"},{"comment":"There is a direct inconsistency in the central quantitative claim. The abstract says ASR 'exceeding 50% across all safety categories'; the introduction reports 52%/67%/50% for critical/dangerous/risky; and the plain averages of Table 2 are roughly 72% critical, 52% dangerous, and 50% risky for the two models combined (Baku alone: 83/60/53; ACT alone: 60/43/47). The paper never states which pooling or weighting produces these numbers, and 50% is not 'exceeding 50%'. With only 20 test sequences per scenario and no confidence intervals, rates like 0.1 and 0.3 are statistically weak. Please report per-scenario counts, exact aggregation, and uncertainty intervals.","section":"Abstract, Sec. 1, Table 2"},{"comment":"The real-world experiment applies the attack to collected vision-action sequences and replays them on a physical UR3 arm. This is open-loop: it does not run the perturbed images through the ACT policy online, so it shows that the recorded actions are executable, but not that the closed-loop policy would produce those unsafe actions when receiving adversarial frames. The claim of 'real-world impact' in the abstract and Sec. 9 is stronger than this evidence supports. Please either run the attack in a closed-loop experiment or explicitly describe the validation as replay of attacked trajectories.","section":"Sec. 6.3"},{"comment":"The attack-leader labels and the ASR metric are both generated from the same threshold rules (Eqs. 5-7) without any external annotation or standard calibration. The benchmark is therefore internally consistent by construction, but it cannot independently validate the proposed safety definitions or establish that the measured violations correspond to real hazards. Please provide an independent validation step, such as human/expert annotation of violation severity or a documented calibration against ISO 15066 recommended separation distances and speed limits, or state this limitation explicitly.","section":"Secs. 4.2-4.3, 5.3"},{"comment":"Several load-bearing implementation details are missing: the PGD budget epsilon, number of iterations N and step size alpha in Algorithm 1; the architecture/hyperparameters of the attack leader besides lambda=0.5; and the adaptive sparse-attack threshold. Additionally, Sec. 6.1 trains a dedicated attack leader per scenario and Sec. 8 admits that the framework 'struggles to generalize to unseen tasks'. This does not invalidate the results, but it significantly narrows the scope of the 'systematic study' claim. Please add the missing parameters and move the generalization caveat into the abstract/contributions.","section":"Algo. 1, Sec. 5.2-5.3, Sec. 8"}],"minor_comments":[{"comment":"Typos and language errors should be corrected ('Tybe C', 'that it', 'inference the attack', 'EAI systems-a robot-by'), and the notation in Eq. (1) should be made consistent (phi vs. Phi).","section":"Throughout"},{"comment":"Fig. 5a plots ASR and AD together despite different scales; please use separate axes or normalized values. The AC/AD columns in Table 2 lack units and a caption-level statement of the 'lower is better' convention.","section":"Fig. 5a, Sec. 4.2"},{"comment":"The 'first systematic study' claim would be easier to evaluate with a direct comparison table against cited works (e.g., [11], [73], [76]), listing threat model, attack surface, safety metric, and whether physical execution is demonstrated.","section":"Sec. 7"},{"comment":"The Mahalanobis ratio in Eq. (8) is difficult to parse; the placement of the norm and the exponent should be cleaned up.","section":"Eq. (8)"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for the robotics/AI-security community and the proposed benchmark could become a useful resource. The main risk is that the headline quantitative claims are tied to unspecified thresholds and inconsistent aggregation; these are fixable in revision. The 'first systematic study' wording should also be checked against prior work during revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing you should know: this is a solid, useful paper for the embodied-AI security crowd, not a breakthrough. The genuinely new parts are the ISO/TS 15066-based taxonomy of safety violations (critical/dangerous/risky), the ANNIE-Bench benchmark of 2,400 video-action sequences, and the attack-leader-plus-PGD pipeline that decomposes a long-horizon attack goal into frame-level targets. Prior VLA attacks mostly aimed at accuracy or trajectory degradation; targeting physical safety constraints is a real shift. The paper is honest about its own limits: the attack leader is supervised, generalization to unseen tasks is acknowledged as a weakness, and the real-world experiment is clearly described.\n\nWhat it does well: the benchmark design is concrete and reusable. The evaluation shows a consistent pattern—Baku is more vulnerable than ACT, and the explanation via Min-Max vs Mean-Std normalization is plausible and checks out. The sparse/adaptive attack analysis adds practical value.\n\nThe soft spots are real and load-bearing. The safety thresholds in Eqs. 5-7—Tcritical, the two velocity limits, the forbidden-object sets—are never given numeric values. The attack labels and ASR are defined by those thresholds. Without them, a reader cannot tell whether a 70% ASR means genuine hazard or a threshold inside the normal reach envelope. The aggregation is also inconsistent: the abstract says >50% across all categories, the introduction says 52/67/50, and Table 2's row averages are roughly 72/52/50 with one cell at 0.1. You need the exact filtering rule. This is a fixable reporting problem, not a fatal flaw, but it has to be fixed before the headline claims are credible.\n\nSecondary point: the physical UR3 experiment replays attacked vision-action sequences open-loop rather than running the attack through the live policy. That shows the actions are physically executable, not that a closed-loop attack works. Worth stating as a limitation, which the paper doesn't do.\n\nThe circularity concern from the reader is softer than it looks: the attack leader is trained on labels from the same threshold rules used to compute ASR, so the benchmark can't independently validate the safety definitions. But it doesn't need to—the taxonomy is a design choice, and the internal consistency lets you compare attacks and models. Not a flaw in the attack, just a limit on what the numbers mean.\n\nBottom line: this is a paper that deserves a serious referee. The core contribution—a reusable benchmark and attack pipeline for safety violations in VLA robots—is valuable to the subfield. I'd send it to peer review with a clear request to report the thresholds, reconcile the aggregation, and soften the abstract. It's citable now for the benchmark and taxonomy, but the headline ASR numbers should be quoted with caution until the details are out.","headline":"Useful ISO-grounded safety benchmark and attack pipeline for VLA robots, but the headline ASR numbers are unverifiable until the safety thresholds and aggregation rule are reported.","tokens_in":22512,"tokens_out":1934,"would_cite":true,"duration_ms":18517,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adversarial camera perturbations can make VLA robots violate physical safety constraints in most tested trials.","keywords":["adversarial attacks","embodied AI safety","vision-language-action models","robot manipulation","ISO/TS 15066","safety violation taxonomy","projected gradient descent","benchmark"],"falsifier":"Fix the thresholds to explicit SI values drawn from ISO/TS 15066 (e.g., a specific stopping distance and speed limit), run the same ANNIE-Attack, and compare per-category attack success rate against clean, unattacked rollouts under identical thresholds; if clean rollouts already violate the chosen thresholds at comparable rates, the reported vulnerability is an artifact of threshold placement, and if attacked ASR collapses to the clean baseline, the claimed transferable vulnerability is not robust.","tokens_in":21605,"feed_emoji":"🤖","tokens_out":5605,"duration_ms":53080,"temperature":0.7,"pith_summary":"Embodied AI robots increasingly act through vision-language-action (VLA) models, where camera images map directly to motor commands. This paper argues that safety failures in such systems should be judged not by task accuracy but by physical constraints borrowed from human-robot collaboration standards: minimum separation distance from humans when handling dangerous tools, speed limits, and forbidden collisions. To make that argument concrete, the authors build a benchmark of nine tabletop manipulation scenarios with 2,400 video-action sequences, and an attack framework that uses a trained 'attack leader' to turn a high-level unsafe goal into per-frame perturbation targets, then applies projected gradient descent to the input images. On two recent VLA policies, ACT and Baku, the attack pushes the robot into safety violations in more than half of trials across all three severity levels, and replaying attacked trajectories on a physical arm moved a knife toward a human in 4 of 10 trials. The stakes: if true, any deployment that uploads unauthenticated camera frames to a learned policy is a viable attack surface for physical harm.","feed_headline":"Tiny image tweaks make robots break safety rules in half of trials","feed_subtitle":"Attackers need only alter uploaded camera frames to drive unsafe motion in simulated and physical robots","key_machinery":"Attack leader model: a small network that encodes the current visual observation (two camera views) and the requested attack category into a continuous attack embedding, then predicts a four-dimensional attack direction ({−1,0,1} per action channel) and a scalar attack scale. It is trained on the paper's Tibbers dataset with a cross-entropy direction loss plus a mean-squared scale loss. Its work in the argument is to solve the missing-label problem of video action attacks: instead of choosing arbitrary target actions frame by frame, the leader maps the long-horizon goal 'cause a critical/dangerous/risky violation' into a concrete action delta; that delta is added to the clean action to form","core_discovery":"The central claim is that adversarial safety attacks on embodied AI are both definable and practically effective. The paper defines safety as three constraints derived from ISO/TS 15066 — a critical distance separating dangerous tools from humans, velocity limits on the end effector and manipulated objects, and a rule that contact never includes forbidden objects — and classifies violations into critical, dangerous, and risky. It then introduces ANNIE-Attack: an attack leader network that takes the current observation and the chosen violation category, and outputs a direction and scale in action space; adding that delta to the model's clean action gives a frame-level target, and PGD optimize","pith_inferences":["Beyond the paper's experiments, the safety rules become an enforceable test harness only if the thresholds in Eqs. 5–7 are fixed as concrete numeric values; an obvious follow-up is an ISO-derived threshold table that external auditors can apply.","The attack leader's per-scenario training suggests a scaling path the paper does not explore: a leader trained across many tasks may transfer to zero-shot scenarios, turning this from a benchmark-specific attack into a general policy vulnerability.","The reported normalization sensitivity implies a cheap defensive experiment — re-normalizing action outputs (z-score style) before execution and measuring ASR drop — that the paper does not run.","If regulators adopt physical-safety success criteria, the attack success rate under bounded perturbations could serve as a certification-style metric for VLA deployments."],"forward_implications":["Any VLA robot that accepts camera frames over an untrusted channel — storage, memory, or network — inherits the attack surface; cloud offloading makes the threat realistic.","Safety-aware evaluation should replace task success rate with constraint-based metrics, and the paper's ASR/AC/AD/TSRC tuple gives a concrete starting point.","Action normalization matters for robustness: Min-Max normalized policies (Baku) amplify small perturbations, while mean-std normalization (ACT) dampens them, so normalization choice is a defensive lever.","Sparse adaptive attacks undermine the assumption that attackers need continuous frame access; ANNIE-ADAP attacks roughly every 3 frames on average yet keeps perfect ASR in the reported scenario.","Real-world transfer is possible: attacked trajectories replay on a physical UR3 arm, moving a held knife toward a human in 4 of 10 trials."],"supporting_citations":[{"why":"Supplies ISO/TS 15066 collaborative-robot safety modes from which the paper derives its three safety constraints.","marker":"[28]"},{"why":"ACT is one of the two VLA policies attacked; provides the bimanual manipulation policy and training setup.","marker":"[95]"},{"why":"Baku is the second attacked VLA policy; its Min-Max action normalization is used to explain higher attack success.","marker":"[20]"},{"why":"Projected Gradient Descent is the underlying adversarial optimizer that produces frame-level perturbations from the attack leader's targets.","marker":"[45]"},{"why":"ResNet image encoders are the backbone of the attack leader model, extracting visual features from the two camera views.","marker":"[23]"},{"why":"ManiSkill simulator underlies ANNIE-Bench, providing physics and rendering for the nine scenarios.","marker":"[66]"}],"fun_headline_variants":["Image tweaks trip 50% of robot safety tests","Robots fail half of safety tests under image attacks","Slight image changes trigger robot safety violations in 50% of runs","Attack on robot vision yields safety breaches in over half of trials","Fifty-percent failure in robot safety after adversarial images"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The safety thresholds in the three constraint equations — Tcritical, the two velocity limits, and the forbidden-object set — are never assigned numeric values, so every attack success rate in the paper counts violations against rules the reader cannot independently check.","fun_headline_variants_meta":{"raw":{"variants":["Image tweaks trip 50% of robot safety tests","Robots fail half of safety tests under image attacks","Slight image changes trigger robot safety violations in 50% of runs","Attack on robot vision yields safety breaches in over half of trials","Fifty-percent failure in robot safety after adversarial images"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000793,"raw_usage":{"total_tokens":3364,"prompt_tokens":810,"completion_tokens":2554,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":2484}},"tokens_in":554,"tokens_out":2554,"duration_ms":18999,"temperature":1.0,"reasoning_tokens":2484,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T10:55:36.930004+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Fix the thresholds to explicit SI values drawn from ISO/TS 15066 (e.g., a specific stopping distance and speed limit), run the same ANNIE-Attack, and compare per-category attack success rate against clean, unattacked rollouts under identical thresholds; if clean rollouts already violate the chosen thresholds at comparable rates, the reported vulnerability is an artifact of threshold placement, and if attacked ASR collapses to the clean baseline, the claimed transferable vulnerability is not robust.","supporting_citations":[{"cited_title":"Maniskill3: Gpu parallelized robotics simulation and rendering for generalizable embodied ai, 2024","cited_arxiv_id":null,"evidence_quote":"ManiSkill simulator underlies ANNIE-Bench, providing physics and rendering for the nine scenarios."}],"review_version":1}