{"id":"46a2c63f-e4f4-481d-9ec4-2a191fa70feb","arxiv_id":"2608.10393","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"A diffusion-based attack creates natural-looking patches that steer vision-language-action robot policies toward attacker-chosen actions in white-box and action-only black-box settings.","lead":"This paper shows a way to fool AI-powered robots with ordinary-looking printed patterns, making them freeze or move wrongly. The attack works even without knowing the robot model's internals, and succeeds on a real robot arm, signaling a safety risk for deployed robotic systems.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"ASR is rollout-level task failure, not target-action consistency; the only direct evidence of targeted steering (AP) is relegated to an appendix and not compared against baselines, leaving the headline 'steering' claim under-supported.","rationale":"The reader's weakest assumption pinpoints exactly the same load-bearing concern: the headline ASR metric conflates task failure with targeted steering. The paper's own definition of AP (Eq. 11) and its acknowledgment that AP 'separates targeted control from incidental performance degradation' make this the critical hinge of the central claim. If the high ASR values were largely driven by out-of-distribution collapse rather than the attacker-specified action, then the advertised contribution — controllable, natural-looking patches that steer VLA policies to target behaviors — would be substantially overstated, even though the existence of some adversarial effect would remain. The paper does provide some direct evidence for targeted behavior: Appendix C reports DURA's AP values (82.0% white-box, 67.3% black-box) and a qualitative real-robot toggle demonstration in Figure 3. However, AP is not integrated into the main results, no baseline AP is reported, and the matching criterion is undefined, so the quantitative case is incomplete. The proposed check — computing AP for all target-only methods with a declared matching threshold — would settle whether the ASR gap is accompanied by a corresponding AP gap. If AP tracks ASR and exceeds baselines, the claim stands; if not, the verdict should move toward REJECT or at least require the AP evidence before ACCEPT. The reader's CONDITIONAL verdict is therefore appropriate; my stress-test does not change it, hence UNCHANGED.","tokens_in":15036,"tokens_out":6903,"duration_ms":72139,"concrete_test":"Re-run the Table 1 and Table 2 evaluation pipelines for all target-only methods (DURA, TMA, TMA-NES, and benign/clean-patch conditions) and compute AP (Eq. 11) alongside ASR, using an explicitly stated action-matching criterion (e.g., L2 distance ≤ 5cm for translation and ≤ 5° for rotation, or exact token match for discrete actions). Report AP and ASR jointly per suite and per access setting. If DURA's AP is not substantially higher than the next-best target-only baseline's AP (e.g., >15 points), or if DURA's AP trails its own ASR by more than 20 points, the conclusion that high ASR reflects targeted steering rather than generic task failure is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that DURA 'steers the robot toward attacker-specified target actions' (Abstract). But the headline metric, ASR, is defined in Eq. 10 as the fraction of rollouts in which the policy fails to complete the task — an inverse task-success rate, not a measure of whether the executed behavior matches the attacker-chosen action. The paper itself acknowledges this gap: Eq. 11 defines Attack Precision (AP) as the per-step fraction of actions matching the target, and the main-text claim that failures 'stem from the intended target action rather than incidental out-of-distribution degradation' rests entirely on AP. Yet AP is only reported in Appendix C, for DURA alone, with no per-baseline AP, no AP-ASR joint table, and no explicit action-matching threshold. Under white-box access, average AP is 82.0% while ASR is 100%; under black-box, AP is 67.3% while ASR is 79.3%. Without knowing how 'matching the target action' is operationally defined, and without comparing AP against baselines that also achieve high ASR, the 79.3-100% ASR numbers could reflect generic OOD collapse (e.g., the policy freezing or flailing) rather than controllable, attacker-specified steering. The real-robot Franka result is also qualitative (Figure 3), so it cannot quantify how often the printed patch induces the target behavior. This is not a dispute about the existence of an attack, but about whether the evidence supports the stronger, advertised property of targeted control.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces DURA, a diffusion-based unrestricted adversarial patch attack for vision-language-action (VLA) models. The method optimizes a single localized patch along the latent trajectory of a pretrained diffusion model, using a clean anchor trajectory to preserve visual naturalness while a VLA loss steers the policy toward an attacker-specified target action. DURA supports white-box backpropagation and black-box action-output estimation, and is evaluated on OpenVLA and π0-FAST across LIBERO simulation suites and on a real Franka arm. The paper reports 79.3–100% attack success rates (ASR), improved visual naturalness relative to pixel-space baselines, and a correct score-function black-box estimator derivation in Appendix A.","tokens_in":15404,"tokens_out":4086,"duration_ms":47188,"significance":"If the targeted-steering claim is substantiated, DURA would be a practically important result: a printed, visually natural patch could repeatedly induce attacker-chosen robot behaviors in deployed VLA systems, including under a realistic black-box threat model. The method is clearly specified, the black-box estimator derivation in Appendix A is a legitimate REINFORCE/score-function estimator, and the evaluation includes multiple baselines, two victim models, robustness to input transformations, and a physical-robot demonstration. The main weakness is that the headline metric, ASR, measures task failure rather than target-action consistency; the only direct evidence of targeted steering, Attack Precision (AP), is relegated to Appendix C and is not compared against baselines. This gap is load-bearing for the central claim and needs to be closed before the results can be accepted as stated.","major_comments":[{"comment":"The headline metric ASR is defined as the fraction of rollouts in which the policy fails to complete the task. This is a task-failure rate, not a direct measure of whether the executed behavior matches the attacker-specified target action. The abstract and conclusion claim that DURA 'steers the robot toward attacker-specified target actions,' but the 79.3–100% ASR numbers alone only show that the patch disrupts task completion. Because the central contribution is targeted control rather than generic disruption, the main results should jointly report ASR with a target-matching metric such as AP, or otherwise disentangle steering from out-of-distribution degradation.","section":"Evaluation Metrics, Eq. (10)"},{"comment":"Attack Precision (AP, Eq. 11) is the only metric that separates targeted steering from incidental failure, yet it appears only in Appendix C, is reported only for DURA, and is not compared against any baseline. A reader cannot tell whether UADA, TMA, or TMA-NES—which also achieve high ASR—would also achieve high AP under the same target-action definition. Without per-baseline AP or an ASR–AP joint table, the claim that DURA's failures 'stem from the intended target action rather than incidental out-of-distribution degradation' (Main Results) is not supported by the presented evidence.","section":"Appendix C vs. Main Results"},{"comment":"The clean patch baseline, which is just a normal-looking patch inserted without any optimization, already reaches 67% ASR on LIBERO-Long in the physical patch setting. DURA reaches 74% ASR there in the black-box setting, an improvement of only 7 percentage points over patch insertion alone. This suggests that on this suite, much of the measured 'attack success' may be due to the patch itself being out-of-distribution, not to target steering. The paper should report clean-patch AP and per-suite breakdowns so reviewers can assess the marginal effect of the optimized patch.","section":"Table 1, LIBERO-Long physical row"},{"comment":"The physical Franka result is qualitative: a single illustrative rollout in which the patch causes the arm to freeze and then resume when the patch is removed. The text states the attack is 'controllable and repeatable on real hardware,' but no quantitative counts, number of trials, success rates, or per-step action-match statistics are provided. Given that the physical setting is central to the paper's deployability claim, the authors should either report quantitative real-robot AP/ASR over multiple trials or soften the claim accordingly.","section":"Real-Robot Results, Figure 3"}],"minor_comments":[{"comment":"The definition of AP says a step is counted if it matches the target action '(e.g., no-action or move-forward)', but no operational threshold or distance criterion is given. Please specify exactly how 'matching the target action' is determined, including tolerance for continuous action dimensions.","section":"Eq. (11)"},{"comment":"The Benign and Clean patch rows are identical under the 'Simulated' and 'Physical' columns. Please clarify what differs between these two settings; as presented, it looks like the same numbers are duplicated.","section":"Table 1"},{"comment":"The phrase 'over100trials' is missing spaces, and the appendix reports a single run without error bars. Given the 61–99% AP range across suites, seed variance should be reported to support the claim of consistent steering.","section":"Appendix C"},{"comment":"Some reference entries contain formatting artifacts, such as 'PMLR.' at the end of the OpenVLA entry. Please proofread the reference list and ensure consistent formatting.","section":"References"},{"comment":"The radar plots in Figure 11 are visually dense; reporting the numeric AP values in a table would make the robustness claim more precise and easier to verify.","section":"Appendix E"}],"recommendation":"major_revision","confidential_remarks":"The method is technically sound and the direction is timely, but the paper currently overstates the targeted-control claim relative to the evidence. The missing AP comparison against baselines and the high clean-patch ASR on LIBERO-Long are the two issues that block acceptance in my view. Both are addressable with additional experiments that are within the scope of the current study, so I recommend major revision rather than rejection. I did not find evidence of unfair baselining, though I would ask the authors to double-check that the query budgets and optimization steps for TMA-NES are indeed matched, since the efficiency comparison in Figure 7 depends on it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nMy take on 2608.10393: it is a solid empirical contribution and deserves a serious referee, but the central claim is currently over-sold. What is genuinely new is the combination: diffusion-prior patch synthesis with a black-box action-output estimator (a REINFORCE/score-function gradient) and joint multi-frame optimization. The components are known, but the combination for VLA models with action queries is real, and the derivation in Appendix A is correct. The authors also run a wide set of experiments—two VLA models, LIBERO, a physical Franka, naturalness metrics, query budgets, input transformations—and the attack does outperform baselines on ASR.\n\nThe soft spot is the gap between the headline metric and the advertised behavior. ASR is defined as rollout-level task failure; it measures disruption, not targeted steering. The paper's own direct evidence for steering is AP, reported only in Appendix C, for DURA alone, with no baseline comparison and no explicit action-matching threshold. That leaves the \"steering\" claim under-supported: the high ASR numbers could partly reflect out-of-distribution collapse rather than controllable target actions. The real-robot demo is qualitative, and the main tables have no error bars. Also, in LIBERO-Long physical, the clean patch already reaches 67% ASR, so the incremental claim is narrower than it initially looks. These are fixable issues, not fatal flaws.\n\nThe attack is real and the method is sound; the presentation just needs to match the evidence. I would send this to peer review, but with a request for major revision: promote AP to a headline metric, compare it against baselines, define the matching rule, add error bars, and clarify what the physical experiments actually quantify.\n\nFor someone working on VLA safety or adversarial robustness, this is worth a careful look. I would not cite it in the next year until the AP numbers are in the main text, but I would bring it to a reading group.\n\nRecommendation: accept for review with major revisions.","headline":"Solid attack paper with a real method, but the headline ASR overstates targeted steering; AP should be front and center.","tokens_in":15916,"tokens_out":2090,"would_cite":false,"duration_ms":22656,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A natural-looking printed patch can steer a Vision-Language-Action robot to attacker-chosen actions, even with no access to the model's internals.","keywords":["vision-language-action models","adversarial patches","diffusion models","black-box attack","robotic manipulation","physical adversarial attack","target action steering","unrestricted attack"],"falsifier":"Recompute the paper's headline ASR results using attack precision as the primary metric on the exact settings of Tables 1 and 2: if ASR remains high while AP stays near the clean-patch baseline (19–32% reported), the attacks are mostly breaking the policy rather than steering it, and the central claim of targeted control is not supported.","tokens_in":14854,"feed_emoji":"🤖","tokens_out":8418,"duration_ms":80841,"temperature":0.7,"pith_summary":"This paper claims that a visually natural printed patch, placed in a robot's camera view, can reliably steer a Vision-Language-Action (VLA) policy toward an attacker-specified target action. The proposed method, DURA, searches over patch content along the latent trajectory of a frozen diffusion model, so the patch stays natural-looking while the VLA loss pushes behavior toward the target. DURA works in both white-box settings and black-box settings where only predicted actions are visible, achieving 79.3–100% attack success across OpenVLA and π0-FAST on LIBERO and transferring to a real Franka arm. If true, deployed robots can be controlled by an ordinary-looking object in the scene, without modifying the model, instruction, or robot state. The paper argues this exposes a physical safety risk that defenses must address.","feed_headline":"Natural-looking patches hijack robots to attacker-chosen actions","feed_subtitle":"No model access needed: a diffusion-crafted sticker can freeze a robot or move it as the attacker wants.","key_machinery":"The central object is the diffusion-mixed latent trajectory: a clean anchor trajectory precomputed by DDIM from a benign seed patch, and an adversarial trajectory that is softly tied to it at each step by mixing latents before each DDIM denoising step. The attack update is applied as $z_{t-1} = u_{t-1} - s \\cdot g_t$, where $g_t$ is either the backpropagated VLA target loss gradient (white-box) or a score-function estimator over queried action losses (black-box). This keeps the decoded patch near the natural image manifold while the VLA objective shapes its content, which is what lets the patch be both natural and action-steering.","core_discovery":"The central claim is that unrestricted, natural-looking adversarial patches can be crafted for VLA models by optimizing in the latent space of a pretrained diffusion model, alternating DDIM denoising steps with an attack update that steers the decoded patch toward a target action. In the white-box setting the update is the gradient of the target-action loss through policy and decoder; in the black-box setting it is a score-function estimate computed from action-output queries alone. The same patch, optimized over diverse frames and instructions, transfers across scenes and, when printed, induces repeatable attacker-chosen behaviors on a physical Franka arm. The result is framed as turning the workspace itself into an attack surface.","pith_inferences":["A consequence the authors leave implicit is that the same latent-trajectory recipe may transfer to other multimodal policies beyond manipulation; if so, the physical attack surface extends to any camera-driven vision-language system.","The headline ASR metric counts any task failure as an attack success; the paper's own Attack Precision (AP) numbers live in Appendix C. A reader who wants the targeted-control claim verified should ask for AP on the main table's settings, since high ASR with low AP would mean the patch mostly degrades the policy rather than steering it.","A testable extension the paper does not run: vary the anchor weight $\\alpha_w$ and the partial-noise timestep $t_{\\text{start}}$ and measure both naturalness and AP; the current choice ($\\alpha_w=0.2$, $t_{\\text{start}}=0.5$) may not be optimal for out-of-distribution scenes or unusual target actions.","The robustness results in the appendix suggest JPEG compression at Q=10 slightly lowers AP on OpenVLA; probing whether a larger patch or a stronger attack step recovers that precision would clarify the trade-off between naturalness and robustness."],"forward_implications":["A single printed patch, optimized over diverse frames and instructions, can act as an on-demand switch: inserting it drives the robot toward the target action, removing it lets the robot resume normal behavior, as demonstrated on a real Franka arm.","The attack succeeds without modifying the model, the language instruction, or the robot state; only the camera-visible patch is changed.","The method attains 79.3–100% ASR across OpenVLA and π0-FAST in both white-box and black-box settings, outperforming the compared baselines in the reported experiments.","The attack remains effective under JPEG compression, bit-depth reduction, and Gaussian noise, with ASR staying at 90–100% in the reported settings.","Because the black-box variant requires only predicted actions, closed-source VLA systems are also exposed to this class of physical attack."],"supporting_citations":[{"why":"Primary white-box victim model, OpenVLA-7B, fine-tuned on LIBERO; the main ASR tables evaluate attacks against it.","marker":"(Kim et al. 2024)"},{"why":"Second victim model π0-FAST, whose action-chunk representation tests transfer across VLA architectures.","marker":"(Pertsch et al. 2025)"},{"why":"Supplies the frozen latent diffusion model (VAE encoder/decoder and denoiser) whose latent trajectory constrains patch naturalness.","marker":"(Rombach et al. 2022)"},{"why":"Provides DDIM, used both for the clean anchor trajectory and for each denoising step in the optimization loop.","marker":"(Song, Meng, and Ermon 2021)"},{"why":"LIBERO simulation benchmark on which all four task-suite evaluations are run.","marker":"(Liu et al. 2023)"},{"why":"BridgeData V2 real-world dataset used to train the physical-setting OpenVLA model evaluated on the Franka arm.","marker":"(Walke et al. 2023)"},{"why":"Score-function estimator (REINFORCE-style) that gives the black-box gradient direction from action-output queries.","marker":"(Williams 1992)"}],"fun_headline_variants":["Diffusion patches hijack robots to attacker-chosen actions","Black-box attack: natural stickers steer physical robots","Unrestricted robotic attack via diffusion-crafted patches","Natural-looking adversarial patches control VLA robots","No model access: diffusion patch hijacks robot actions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper treats any rollout that fails the task as an attack success, so the claim that DURA steers the robot toward the attacker-specified action depends on those failures actually matching the target behavior and not just being generic policy breakdowns.","fun_headline_variants_meta":{"raw":{"variants":["Diffusion patches hijack robots to attacker-chosen actions","Black-box attack: natural stickers steer physical robots","Unrestricted robotic attack via diffusion-crafted patches","Natural-looking adversarial patches control VLA robots","No model access: diffusion patch hijacks robot actions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000287,"raw_usage":{"total_tokens":1640,"prompt_tokens":853,"completion_tokens":787,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":469,"completion_tokens_details":{"reasoning_tokens":713}},"tokens_in":469,"tokens_out":787,"duration_ms":15264,"temperature":1.0,"reasoning_tokens":713,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:38:17.759959+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Recompute the paper's headline ASR results using attack precision as the primary metric on the exact settings of Tables 1 and 2: if ASR remains high while AP stays near the clean-patch baseline (19–32% reported), the attacks are mostly breaking the policy rather than steering it, and the central claim of targeted control is not supported.","supporting_citations":[],"review_version":1}