{"id":"0f986c2e-9070-4ecf-aad1-92f06588e035","arxiv_id":"2411.16622","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"On six printed photos, STE-augmented PGD forces 0/6 correct classifications at a 4/255 bound; in CARLA, an 8/255 patch lowers object-detection AP50 to 4.22%.","lead":"This paper shows that a known gradient trick, the straight-through estimator (also called BPDA), can create adversarial image perturbations that survive physical printing and camera capture while staying within small, human-imperceptible pixel bounds. The result matters because it suggests physical-world attacks on computer vision systems can be as potent as digital ones, weakening a common assumption that camera and printer distortions block such attacks.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The central claim depends on an unmeasured sign-accuracy assumption for STE's identity backward pass; the paper's own §3.4 admits pixel misalignment breaks it, and zero-accuracy results (Table 1) may be inflated by unmodeled cropping artifacts.","rationale":"The paper's strongest claim is that STE/BPDA makes ℓ∞-bounded perturbations survive physical distortions, with Table 1 reporting zero accuracy at ϵ = 4/255 and Table 2 reporting AP50 dropping to 4.22% at ϵ = 8/255. For that claim to hold, the identity backward pass must be a reliable sign oracle for the true distortion. Section 3.1 asserts this without proof, and the physical pipeline includes perspective transforms and cropping, which are not pointwise operations; a small misalignment can invert the sign of high-frequency components of a 4/255 perturbation. The authors' own Section 3.4 concedes this, and Figure A.1 shows white cropping edges in the reported images. The CARLA experiments are structurally similar: the residual C in Equation 5 is assumed negligible but not measured, and UE4 lighting and weather can alter patch appearance. I am not claiming the method is fraudulent; the mathematical construction is sound and the empirical direction is plausible. But the central empirical claim rests on a mechanism that is asserted rather than verified. A single controlled sign-agreement measurement would settle it. Because the paper's current evidence is insufficient, the reader's CONDITIONAL verdict is appropriate; no change is needed. The reader identified the same load-bearing assumption, and the additional concerns about sample size, missing baselines, and absent artifacts reinforce, but do not replace, this primary issue.","tokens_in":12968,"tokens_out":7562,"duration_ms":69675,"concrete_test":"Along the Table 1 PGD trajectory for ϵ = 4/255, select a subset of iterations and, for each, print and photograph the current perturbed image plus versions with a single-pixel perturbation added at a random set of, say, 50 coordinates. Use the captured photos to estimate the true sign of the loss change for each coordinate by finite differences, and compare it to the sign produced by STE's identity backward pass at the same δ_t. Report the per-coordinate sign-agreement rate. If the agreement rate is not clearly above chance, or is below the level sign-based PGD needs to make progress, the zero-accuracy result is not attributable to the claimed STE sign-accuracy mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 3.1, the paper asserts (Equation 3) that when a physical distortion d(·) produces a similar image, the identity backward pass yields a gradient whose sign is always accurate, and that this makes STE optimal for sign-based attacks like PGD. This is the load-bearing assumption for both headline results (Table 1 and Table 2). It is never measured. The printout pipeline adds a perspective transform from manual handheld photos and a crop (Section 3.2); after a homography, d(x + δ) is not pixel-wise close to x + δ. A one-pixel misalignment can flip the sign of high-frequency components of a perturbation bounded by ℓ∞ = 4/255. The authors concede in Section 3.4 that any pixel misalignment would break the assumption of STE, and Figure A.1 shows white edges from imperfect cropping on the very images used for the zero-accuracy claim. Thus the Physical column of Table 1 may report an attack that is not the bounded perturbation but an unbounded cropped artifact plus a misaligned pattern. The CARLA experiments (Equation 5) share the same structure: the detached residual C between the UE4 renderer and the differentiable perspective renderer is assumed small but never quantified, despite lighting and weather altering patch appearance. Without a measurement of sign agreement between the identity backward pass and the true distortion, the claim that STE forces zero accuracy is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes using a straight-through estimator (STE, also called BPDA) to generate ℓ∞-bounded adversarial examples that survive non-differentiable physical distortions. In the global perturbation threat model, the method applies the true non-differentiable distortion in the forward pass and an identity backward pass, then uses FGSM or PGD on the resulting surrogate gradient. Experiments with printed ImageNet images report zero classification accuracy for PGD at ℓ∞ = 4/255 and for FGSM at ℓ∞ = 16/255. In the patch threat model, STE is combined with a differentiable perspective renderer in the CARLA simulator; the paper reports AP50 dropping from 43.29% to 4.22% for an ℓ∞ = 8/255 patch and to 0% for an unbounded patch. The authors claim these are the first imperceptible, physically realizable adversarial examples with small ℓ∞ bounds that force near-zero accuracy.","tokens_in":13227,"tokens_out":5320,"duration_ms":53012,"significance":"If the empirical claims hold, this is a significant result: it would show that small ℓ∞-bounded perturbations, imperceptible to humans, can survive print-and-camera pipelines and simulator-based rendering, contradicting the common assumption that physical distortions act as a defense. The mathematical formulation is clean and the stop-gradient identities in Eq. (3) and Eq. (5) are exact. The use of an external pre-trained ResNet-50 and a public CARLA detection model, rather than a model trained by the authors, is a strength, as is the internal consistency of the reported trends: larger budgets monotonically reduce accuracy in nearly all rows. However, the central claims rest on an unmeasured sign-fidelity assumption for the identity backward pass, on six images with no repeated trials, and on CARLA scenes that were recreated rather than taken from the official GARD dataset. The significance is therefore conditional on additional measurement and control experiments.","major_comments":[{"comment":"The assertion that 'the sign of the approximate gradient is always accurate' is load-bearing for the zero-accuracy claim but is never measured. The argument only requires d(x + δ) to be similar to x + δ; for sign-based PGD, one needs per-coordinate sign agreement between the identity backward pass and the true distortion. The printout pipeline in Section 3.2 applies a perspective transform from handheld photos and a manual crop, so after the homography d(x + δ) is not pixel-wise close to x + δ. A one-pixel misalignment can flip the sign of high-frequency components at ε = 4/255. The paper itself concedes in Section 3.4 that any pixel misalignment breaks the assumption, and Figure A.1 shows white edges from imperfect cropping on the images used for the zero-accuracy claim. Please quantify the sign agreement between the identity backward pass and the true distortion (for example, by finite differences on registered images), and either eliminate cropping artifacts or report results separately for cleanly registered images. Without this, the Physical column of Table 1 may reflect misalignment plus cropping artifacts rather than the bounded perturbation alone.","section":null},{"comment":"The detached residual C = c(p(x + δ), e) - r(x + δ) is assumed small because the differentiable renderer places the patch at the same location as the UE4 renderer, but this assumption is never quantified. UE4 applies lighting, weather, materials, and occlusion, while the differentiable renderer is only a perspective transformation. Furthermore, Section 4.2 states that the GARD scenes had to be recreated with the OSCAR toolkit because the released CARLA metadata was incomplete, so the evaluation is not on the official GARD images. Please report the distribution of pixel residuals between the UE4 render and the differentiable renderer across the 20 scenes (for example, mean and maximum absolute difference per patch), and ideally include a comparison with digitally composed patches as in the original GARD setup. If C is not small, the 4.22% AP50 result may be an artifact of renderer mismatch rather than evidence that the bounded perturbation survives the full pipeline.","section":null},{"comment":"The headline zero-accuracy result is based on six images with no repeated trials. The 113 printouts described in Section 3.2 correspond to one print per condition, so each row in the Physical column is a single observation per image. The claim that STE 'forces zero accuracy' therefore rests on 6 correct classifications becoming 0, a very small sample. Please report per-image results, provide confidence intervals or repeated print/capture trials, and include a control with random bounded perturbations (or an unprinted digital perturbation) to confirm that the effect is attributable to the optimized perturbation under the stated ε bound rather than to cropping, perspective, or printing artifacts. The visible white edges in Figure A.1 make this control particularly important.","section":null},{"comment":"The digital and physical columns of Table 1 are surprising in a way that needs explanation: for PGD + STE at ε = 4, the digital accuracy is 3/6 but the physical accuracy is 0/6, meaning the attack is stronger after the print-and-camera distortion than in the digital domain. If STE is only a backward-pass approximation, one would expect physical effectiveness to be no better than digital effectiveness. The current text addresses the gap between digital PGD and STE-physical PGD, but not the comparison between STE-physical and STE-digital. Please measure the realized perturbation in the captured image after registration (for example, the ℓ∞ difference between the registered photo of the benign printout and the attacked printout) and discuss whether the physical distortion itself contributes to the drop. Without this measurement, it is difficult to attribute the zero-accuracy result to the small ℓ∞-bounded perturbation rather than to unintended artifacts.","section":null}],"minor_comments":[{"comment":"The notation 'where C is a constant' is misleading because C = d(x + δ) - (x + δ) depends on x and δ; it is a term detached from differentiation, not a constant in the mathematical sense. Please rephrase as 'a term treated as constant by the stop-gradient operation.'","section":null},{"comment":"The terms 'global perturbation threat model' and 'patch perturbation threat model' are used without definition until Sections 3 and 4. Please define them at first use, since they are central to the claimed contributions.","section":null},{"comment":"The caption refers to an 'empirical threshold of loss' that yields zero accuracy, but does not explain how this threshold was computed. Please describe the procedure used to set the dashed cyan line.","section":null},{"comment":"The TIDE error columns (Class, Box, Other, Dup., Bg., Missed) lack a clear explanation of what each count represents and how the columns relate to the TIDE taxonomy. Please add a sentence or footnote explaining these counts.","section":null},{"comment":"The row labels in Figures A.1 and A.2 appear to contain a typo: the third data row is labeled 'hot pot' in both figures, but the text describing the setup lists the six images as including 'ladle' and only one 'hot pot'. Please correct the labels.","section":null},{"comment":"References [36] and [37] have a dangling comma before 'Accessed' in the bibliographic entries; please clean up the formatting.","section":null}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for a computer-vision or security venue, and the mathematical core is sound. The main risk is that the physical-world claims are stronger than the evidence: the sign-fidelity assumption of STE is unmeasured, the printout experiments use a very small sample with visible cropping artifacts, and the CARLA evaluation uses recreated scenes. These issues are addressable experimentally, so I do not recommend rejection. However, the revision should add quantitative measurements of the STE gradient sign accuracy, clean-registration controls, repeated trials, and a quantification of the CARLA renderer residual before the zero-accuracy and 4.22% AP50 claims can be accepted."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a clean application of a known trick (STE/BPDA) to physical-world attacks, and the empirical showings are interesting. But the headline claim—zero accuracy at eps=4/255 in the physical world—rests on an unmeasured assumption and a small, possibly confounded experiment. I'd send it to review, but I wouldn't let the strong conclusion stand without more support.\n\nWhat's genuinely new: using STE with identity backward pass to cross the non-differentiable printer/camera pipeline, and the differentiable-rendering extension for patches. The math is clean—Eq. 3 and Eq. 5 are exact stop-gradient identities—and the authors are honest that STE is BPDA from Athalye et al. The CARLA results show a monotonic drop in AP50 as the budget grows, which is a good sanity check. The loss curves in Fig. 4 show the optimization converging in a way that matches the story.\n\nSoft spots: (1) The central assumption—that the identity backward pass gives a reliable sign for the true distortion—is asserted but never measured. The paper itself concedes in §3.4 that any pixel misalignment breaks this, and Figure A.1 shows white edges from imperfect cropping on the very images used for the zero-accuracy claim. So Table 1's zero accuracy may be inflated by cropping artifacts or misalignment, not solely by the bounded perturbation. I don't think this is fatal, but it means the physical claim is not as clean as the abstract suggests. (2) The global-perturbation experiment uses six images, no repeated trials, and no released code/data. That's thin for a \"we urge the community to re-evaluate the threat\" conclusion. (3) In the CARLA patch attack, the detached residual C between the UE4 renderer and the differentiable perspective renderer is never quantified. Lighting and weather change the patch appearance, so the approximation error could be significant.\n\nWho this is for: researchers working on physical adversarial examples or defense evaluation. It's a useful demonstration and a good discussion paper, especially for the STE sign-accuracy question. With more experiments, direct measurement of sign agreement, and released artifacts, it could be a solid contribution.\n\nRecommendation: send to peer review, but the authors should be pushed to measure sign accuracy, add repeated trials, release code/data, and compare same-budget non-STE baselines. As is, the strong claims are premature.","headline":"Well-executed application of known STE/BPDA trick to physical attacks; strong claims rest on an unmeasured sign-accuracy assumption and a small, confounded experiment.","tokens_in":13804,"tokens_out":3269,"would_cite":true,"duration_ms":28812,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Straight-through gradient estimation makes imperceptible, L-infinity-bounded adversarial examples work in the physical world, with zero accuracy on printed photos and AP50 down to 4.22 percent in a driving simulator.","keywords":["adversarial examples","physical-world attacks","straight-through estimator","BPDA","imperceptible perturbations","L-infinity bounded perturbations","object detection","autonomous driving perception"],"falsifier":"Print the same six images again with a different printer or camera, generate STE-augmented PGD perturbations at $\\ell_\\infty=4/255$, and count correct predictions of the target classifier; any nonzero correct count would falsify the zero-accuracy claim. A more direct test of the mechanism: for random perturbations, compare the sign of the identity-backward gradient with the sign obtained by finite differences through the physical pipeline; if sign agreement falls well below 100% of pixels, the assertion that the sign is always accurate is refuted.","tokens_in":12759,"feed_emoji":"🖨️","tokens_out":11646,"duration_ms":92253,"temperature":0.7,"pith_summary":"Imperceptible adversarial perturbations, bounded by tiny L-infinity norms, can survive a real printing-and-photographing pipeline and defeat a deployed image classifier; the paper demonstrates this with six printed photos on which a 12-step PGD attack at $\\ell_\\infty = 4/255$ drives correct predictions to zero. The same idea works for patches: a perturbation of $\\ell_\\infty = 8/255$ placed on a billboard-like patch in a photorealistic urban driving simulator drops object-detection AP50 from 43.29% to 4.22%. The enabling trick is a straight-through estimator (STE, also known as BPDA): the exact non-differentiable distortion runs in the forward pass, and the identity function replaces its gradient in the backward pass. Because FGSM and PGD only use the sign of the gradient, the paper argues, the magnitude error from this approximation does not hurt the attack. If the results hold, physical-world adversarial examples are as practical as digital ones, and vision systems in security-sensitive settings should be assessed against this threat.","feed_headline":"Imperceptible perturbations survive printing and force zero accuracy","feed_subtitle":"Straight-through gradients let tiny L-infinity-bounded perturbations hold up after printing and in driving scenes.","key_machinery":"The central object is the straight-through estimator (STE, also known as BPDA), defined by using the true non-differentiable distortion in the forward pass and the identity in the backward pass. In the global-perturbation setup it is expressed with a stop-gradient operation as $f(d(x+\\delta)) = f(x+\\delta + \\mathrm{sg}[d(x+\\delta)-(x+\\delta)])$, so the backward pass sees $x+\\delta$ with a constant offset. In the patch setup the distortion is written as $g_d(\\delta,x,e)=r(x+\\delta)+\\mathrm{sg}[c(p(x+\\delta),e)-r(x+\\delta)]$, where $r$ is a differentiable renderer and the non-differentiable camera-plus-print pipeline only supplies the forward value. The mechanism works because the sign of the identity-backward gradient is asserted to match the true sign whenever the distortion preserves a similar image, and sign-based attacks such as FGSM and PGD only need the sign.","core_discovery":"The paper's central claim is that non-differentiability in physical imaging is not a barrier to imperceptible adversarial examples. On the paper's own terms: apply the true distortion $d(\\cdot)$ in the forward pass, so the loss is exact, and backpropagate through the identity, so the gradient is approximate but cheap. The stop-gradient identity $f(d(x+\\delta)) = f(x+\\delta + \\mathrm{sg}[d(x+\\delta)-(x+\\delta)])$ makes this concrete. For the patch threat model, the same construction uses a differentiable renderer $r(x+\\delta)$ in place of the non-differentiable renderer, with the residual detached as a constant. With this machinery, PGD bounded by $\\ell_\\infty=4/255$ achieves zero accuracy on the printout photos, and an unbounded STE-augmented patch reaches 0% AP50 while an $\\ell_\\infty=8/255$ patch reaches 4.22% AP50 in the simulator.","pith_inferences":["A testable prediction follows from the sign-accuracy mechanism: the attack's effectiveness should degrade as physical distortion becomes more nonlinear or misaligned, so printers with stronger color shifts or cameras with larger perspective error should require larger $\\epsilon$ to reach zero accuracy.","The zero-accuracy result rests on six images and one printer-camera pair; if the threat model is as general as the paper suggests, the same STE recipe should transfer to other classifier architectures and capture devices, a claim the experiments do not yet cover.","Combining STE with expectation-over-transformation noise, which the paper mentions as future work, should yield physical perturbations robust to viewpoint and lighting variation; the mechanism's design allows this without changing the forward-pass trick.","The same stop-gradient construction could be reused as a diagnostic for defenses: a defense that deliberately breaks the sign correspondence between the digital perturbation and the physical loss would neutralize the attack, pointing to a cheap defensive direction."],"forward_implications":["Physical adversarial examples no longer require large, visibly strange perturbations: with STE, $\\ell_\\infty = 4/255$ perturbations force zero accuracy on printed photos, so an attacker does not need to sacrifice imperceptibility.","Patch-based attacks on object detection become practical: an $\\ell_\\infty = 8/255$ patch drops AP50 from 43.29% to 4.22% in a photorealistic driving simulator, and an unbounded patch reaches 0%.","Because the attack is fast (a single FGSM step at $\\ell_\\infty=16/255$ suffices, and a few PGD iterations at $16/255$ cross the zero-accuracy threshold), physical attacks are cheap to generate once the pipeline is built.","The method extends to 3D camouflage: the paper's differentiable-rendering construction works with mesh renderers, so imperceptible textures on 3D objects are a direct corollary.","Defenses that flag abnormal textures will not catch these examples, since the perturbations remain small and visually benign."],"supporting_citations":[{"why":"Defines the straight-through estimator, the identity-backward-pass technique the attack is built on.","marker":"[4]"},{"why":"Introduces backward-pass differentiable approximation (BPDA) for crossing non-differentiable components in adversarial examples.","marker":"[1]"},{"why":"Supplies the printout-photo experimental protocol and shows digital adversarial examples lose effectiveness after printing.","marker":"[24]"},{"why":"Describes expectation-over-transformation, the earlier physical-robustness approach that this work contrasts with tight bounded perturbations.","marker":"[2]"},{"why":"Provides the projected gradient descent algorithm whose sign-only updates make the STE gradient approximation sufficient.","marker":"[27]"}],"fun_headline_variants":["STE unlocks imperceptible adversarial examples in the physical world","Tiny L-infinity perturbations cause zero accuracy after printing","First imperceptible physical adversarial examples with bounded perturbations","Straight-through estimator: invisible attacks in the real world","Non-differentiable distortions? STE makes physical attacks imperceptible"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the identity backward pass preserves the sign of the true gradient through the physical distortion; if clipping, halftoning, color shifts, perspective misalignment, or game-engine lighting ever make that sign wrong, the digitally optimized perturbation is not what the camera sees, and the zero-accuracy results would not transfer.","fun_headline_variants_meta":{"raw":{"variants":["STE unlocks imperceptible adversarial examples in the physical world","Tiny L-infinity perturbations cause zero accuracy after printing","First imperceptible physical adversarial examples with bounded perturbations","Straight-through estimator: invisible attacks in the real world","Non-differentiable distortions? STE makes physical attacks imperceptible"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000631,"raw_usage":{"total_tokens":2945,"prompt_tokens":1010,"completion_tokens":1935,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":626,"completion_tokens_details":{"reasoning_tokens":1853}},"tokens_in":626,"tokens_out":1935,"duration_ms":15910,"temperature":1.0,"reasoning_tokens":1853,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T12:55:58.394436+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Print the same six images again with a different printer or camera, generate STE-augmented PGD perturbations at $\\ell_\\infty=4/255$, and count correct predictions of the target classifier; any nonzero correct count would falsify the zero-accuracy claim. A more direct test of the mechanism: for random perturbations, compare the sign of the identity-backward gradient with the sign obtained by finite differences through the physical pipeline; if sign agreement falls well below 100% of pixels, the assertion that the sign is always accurate is refuted.","supporting_citations":[{"cited_title":"Obfus- cated gradients give a false sense of security: Circumventing defenses to adversarial examples","cited_arxiv_id":null,"evidence_quote":"Introduces backward-pass differentiable approximation (BPDA) for crossing non-differentiable components in adversarial examples."},{"cited_title":"Ad- versarial examples in the physical world","cited_arxiv_id":null,"evidence_quote":"Supplies the printout-photo experimental protocol and shows digital adversarial examples lose effectiveness after printing."},{"cited_title":"Synthesizing robust adversarial examples","cited_arxiv_id":null,"evidence_quote":"Describes expectation-over-transformation, the earlier physical-robustness approach that this work contrasts with tight bounded perturbations."}],"review_version":1}