{"id":"15c17a25-2225-4e04-bf1c-f541dc8e2d9e","arxiv_id":"2411.16162","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A trimming-based attack that selects the most important pixels or patches via Monte Carlo point-wise evaluation outperforms prior sparse and patch attacks on ImageNet.","lead":"This paper presents a method for crafting sparse and patch adversarial attacks by trimming dense perturbations to the most important pixels, evaluated via Monte Carlo sampling. The approach claims state-of-the-art success rates on ImageNet classifiers and, for the first time, jointly optimizes multiple patch locations and perturbations.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The core unsupported step is the Eq. (6) proxy for point-wise importance; a direct correlation test against retrained per-mask optima is needed before the SOTA claim is credible.","rationale":"I agree with the reader that the TrimStep approximation in Section 3.1 is the weakest load-bearing step. The paper's strongest claims are empirical ('significantly outperforms the state-of-the-art', 'first direct solution') and the method's only selection mechanism is the Eq. (6) proxy. The authors are transparent about the assumptions, but they do not test them directly. Without such a test, the reader cannot distinguish between a method that genuinely solves the mask-selection problem and one that happens to work because ImageNet loss landscapes are smooth enough for pointwise heuristics. The novelty issue with Wei et al. 2022a is checkable and could reduce the 'first' claim, but even if that claim were false the empirical method could stand; the Eq. (6) proxy is what the empirical claims rest on. I recommend keeping the CONDITIONAL verdict, with the concrete test above as a condition for acceptance. No change to the existing verdict is needed, which is why I select UNCHANGED.","tokens_in":9638,"tokens_out":7018,"duration_ms":73223,"concrete_test":"Run a small-scale validation of Eq. (6) on ImageNet: take one model (e.g., ResNet-50), a fixed dense perturbation δ at an intermediate trim step, and a target L0 of 128. Sample 100 masks B; for each, run DPGD to obtain δ^B_s, compute ℓ(M(x+δ^B_s),y), and form the true importance vector Lδs as in Eq. (5) via Monte Carlo over the 100 retrained masks. Compute the proxy vector from Eq. (6) using the same masks and δ. Report the Spearman correlation and top-128 Jaccard overlap between the two rankings, and the ASR difference when the attack uses the proxy-selected mask vs. the true top-128 mask. If the correlation is below about 0.7 or the proxy mask loses more than a few ASR points, the approximation is not faithful and the empirical advantage is not explained.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing assumption is the TrimStep approximation (Eq. 5 vs. Eq. 6). The central claim—simultaneous patch-location/perturbation optimization and state-of-the-art ASR—depends on the top-K set chosen from E_B[ℓ(M(x+B⊙δ),y)·B] being close to the set that would be chosen by the actual objective E_B[ℓ(M(x+δ^B_s),y)·B] and ultimately by Eq. (4). The authors list three assumptions under which this should hold, but provide no evidence that they hold on ImageNet. In particular, if the loss depends on correlated groups of pixels, pointwise marginal importance can rank a pixel highly even though it is useless without its partner, or rank a partner low because it rarely appears without the other. The Monte Carlo estimator (1000 samples, Algorithm 2) also has no variance reporting, and the ASR curves in Figs. 2–5 have no error bars. Because the entire method is a heuristic built on this proxy, and the empirical advantage over baselines is the main support for the central claim, this is the load-bearing point.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes sparse and patch adversarial attacks by starting from a dense adversarial perturbation and iteratively trimming it to a target L0 budget. The central idea is a TrimStep procedure that estimates per-point importance via Monte Carlo sampling over random binary masks and then keeps the top-scoring points (or the highest-scoring patches under a kernel constraint). The method is evaluated on ImageNet across five models (InceptionV3, ResNet50, robust ResNet50, Swin-B, ConvNeXt-B), reporting adversarial success rate as a function of the L0 bound for the sparse attack and for 1x1, 2x2, and 4x4 patch variants. The authors claim state-of-the-art sparse attack results and that their approach is the first direct solution to simultaneous patch-location and perturbation optimization.","tokens_in":9830,"tokens_out":4283,"duration_ms":45315,"significance":"If the empirical claims hold, the paper would provide a practical and general mechanism for jointly optimizing patch locations and perturbations, addressing a recognized gap in patch-based adversarial attacks. The paper has notable strengths: the method is described with explicit algorithms, a reference implementation is promised, and the evaluation covers multiple architectures including robust and transformer-based models. The sparse attack results are consistently better than the compared baselines across the reported settings. However, the central approximation on which the method rests—replacing the posterior mask distribution by the prior mask distribution in Eq. (6)—is presented without a formal bound or empirical validation, and the reported curves lack variance information despite the stochastic components (Monte Carlo sampling and restarts). These issues make the state-of-the-art claim not yet fully supported.","major_comments":[{"comment":"The approximation Lδs = E_B[ℓ(M(x+δ^B_s),y)·B] ≈ E_B[ℓ(M(x+B⊙δ),y)·B] is the load-bearing step of the entire method, but it is only motivated by qualitative assumptions. No bound is given and no experiment checks whether the top-k sets selected by Eq. (6) match those selected by the true objective Eq. (5) or Eq. (4) on a tractable subproblem. Since the authors acknowledge that the approximation is exact only under assumptions that are not verified on ImageNet, the \"significantly outperforms\" claim is not yet backed by a correctness argument. Please provide either a quantitative bound (e.g., in terms of the Lipschitz constant of ℓ∘M and the projection error) or an empirical validation showing that the selected masks closely match the optimal masks on a simplified setting, and report the discrepancy as a function of the L0 gap.","section":"3.1, Eq. (5)-Eq. (6)"},{"comment":"The reported ASR curves are single trajectories with no error bars, confidence intervals, or significance tests. The method has multiple stochastic components: MC=1000 mask samples per trim step and nrestarts=11 restarts. The authors also state that MC samples may be \"sufficient\" and compute Eq. (6) directly, but no criterion for sufficiency is given. Without variance reporting, the claim that the method \"significantly outperforms\" the baselines is not statistically supported. Please report mean and standard deviation (or full error bars) over the 11 restarts and, ideally, over several MC seeds, and indicate the number of images used for each curve.","section":"4.1, Figs. 2-5"},{"comment":"The patch-specific contribution is not compared against any existing patch attack. Figures 2-4 show the proposed patch variants only against sparse baselines and against the paper's own sparse attack; no comparison is made to prior patch attacks such as those of Nemcovsky et al., Chen et al., Wei et al., or Zolfi et al. described in the introduction. Since the paper's claimed novelty is simultaneous optimization of patch locations and perturbations, an evaluation against previous patch attacks on the same models and L0 budgets (or a clear explanation of why such a comparison is not meaningful) is necessary to support that claim.","section":"4, Figs. 2-4"},{"comment":"In the point-wise evaluation loop, BCount starts at zero and is incremented by the sampled mask B. If a point is never selected in the MC samples, BCount[i] remains zero and the expression BLoss/BCount is undefined unless a special case is handled. The pseudocode does not describe such handling, which makes the algorithm as written non-reproducible. Please specify how zero-count points are treated (e.g., excluded from the top-k selection or assigned a default score).","section":"A.2, Algorithm 2"}],"minor_comments":[{"comment":"The caption says \"ConvNextB\" while the text refers to the model as \"ConvNeXtVIT\"; please unify the naming.","section":"4, Fig. 5"},{"comment":"The text has a typo: \"M C= 1000\" should be \"MC = 1000\".","section":"4, Experimental settings"},{"comment":"The notation ∥δinit∥0 = N is stated for the initial perturbation, but the algorithm initializes δbest with a uniform draw in [-1,1]^N; please clarify whether this initialization is always full-support and whether the count is over pixels or scalar channels.","section":"A.1, Eq. (9)"},{"comment":"The padding and pooling operations for overlapping patches are described compactly; providing a small example or a precise definition of MaxPool, SumPool, and Pad would improve reproducibility.","section":"A.2, Algorithm 3"},{"comment":"The Koonce reference is not the standard citation for ResNet50; please cite the original He et al. paper.","section":"References"},{"comment":"The text says ASR \"starting from ϵ0=128\" for IncV3 and RN50, but the curves show the reported points; please clarify whether the curves are monotonic and whether the 100% claim is exact or rounded.","section":"4.1, patch discussion"}],"recommendation":"major_revision","confidential_remarks":"The core method is heuristic but arguably interesting; the major issue is that the central approximation in Eq. (6) is not validated and the empirical claims lack variance reporting. The absence of any comparison to prior patch attacks also weakens the patch-novelty claim. I would encourage the editor to request the additional experiments described in the major comments before considering acceptance. The authorship and scope are standard for a workshop paper; the main concern is technical substantiation, not novelty disclosure."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Honest take: this paper introduces a genuinely new approach to sparse and patch adversarial attacks. Instead of the usual convex relaxations, greedy distortion reduction, or homotopy methods, it starts with a dense PGD perturbation and iteratively trims points based on Monte Carlo estimates of per-pixel importance (Eq. 6). That is a real departure, and it produces consistently higher ASR than SparseFool, GreedyFool, PGD_L0 and Homotopy across five ImageNet models, including a robust ResNet50, Swin-B, and ConvNeXt-B. The patch variant handles overlapping patches with a max-out scheme and is a sensible extension. Code is provided. I believe the experimental results: the margins over baselines are large and consistent, not marginal.\n\nThe soft spots are real but not fatal. The paper explicitly approximates the posterior point-wise importance (Eq. 5) by the prior (Eq. 6) and lists three assumptions under which this is accurate, but gives no direct evidence that those assumptions hold on ImageNet. The MC estimator (1000 samples) and the 11 restarts have no variance reporting; the ASR curves in Figs. 2–5 are single trajectories with no error bars. So we cannot tell whether the SOTA margins are within noise, though the consistency across models suggests they are not.\n\nThe other substantive issue: the claim of being the 'first direct solution' for simultaneous patch location/perturbation optimization needs to be reconciled with Wei et al. 2022a, titled 'Simultaneously optimizing perturbations and positions for black-box adversarial patch attacks,' which is cited but not empirically compared. The authors may have a legitimate distinction (white-box vs black-box), but they need to make it.\n\nWho is this for? Researchers working on sparse or physical adversarial attacks. The method is likely to become a useful baseline. It deserves a serious referee: the heuristic is well-motivated, the experiments are broad, and the limitations can be addressed with added variance analysis, a direct comparison to Wei et al. 2022a, and at least a sanity check of Eq. (6) (e.g., correlation against retrained per-mask optima). I would send it to review.","headline":"New trimming heuristic for sparse/patch attacks that consistently beats baselines on ImageNet, but the load-bearing proxy in Eq. (6) is unvalidated and no variance is reported, so treat SOTA margins as provisional.","tokens_in":10379,"tokens_out":2176,"would_cite":true,"duration_ms":20539,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that sparse and patch adversarial attacks can be solved directly by trimming dense perturbations, and that the result achieves state-of-the-art success on ImageNet.","keywords":["adversarial attacks","sparse adversarial attacks","patch adversarial attacks","point-wise trimming","L0 norm bound","Monte Carlo estimation","ImageNet classification","PGD optimization"],"falsifier":"Take a trained ImageNet model, compute a PGDTrim mask for a given L0 budget, then replace an equal number of the lowest-ranked kept pixels with an equal-size set of medium-ranked pixels chosen for high pairwise correlation with the surviving set, keeping perturbation magnitudes fixed; if the replacement achieves equal or higher adversarial success, the point-wise independence assumption behind the ranking is untenable.","tokens_in":9407,"feed_emoji":"🎯","tokens_out":8758,"duration_ms":70403,"temperature":0.7,"pith_summary":"This paper tries to establish that the combinatorial problem of sparse and patch adversarial attacks can be solved directly by starting from a dense adversarial perturbation and trimming it point by point. It claims to be the first method that simultaneously optimizes the locations and the pixel values of multiple patches of any given shape, and that the same approach also improves over existing sparse attacks. If true, the method trades an exponentially large search over pixel subsets for a tractable rank-and-prune loop, giving attackers a stronger tool and defenders a harder baseline. The reported experiments on ImageNet show state-of-the-art success rates for PGDTrim across multiple architectures and L0 budgets, with 100 percent success at modest budgets.","feed_headline":"Point-wise trimming sets new sparse attack records on ImageNet","feed_subtitle":"Jointly optimizing patch locations and values, it beats four prior sparse attacks on ImageNet.","key_machinery":"The load-bearing mechanism is the TrimStep procedure: it computes a per-pixel importance score by averaging, over Monte Carlo-sampled binary masks, the model's loss on the masked perturbation weighted by the mask, then selects the top-k pixels (or patches). The attack wraps TrimStep in a logarithmic trimming schedule, interleaving PGD optimization with Bernoulli dropout that simulates the binary-mask projection, so the dense perturbation stays robust to the next trim and the L0 gap per step stays small. This converts the combinatorial mask search into a sequence of top-k selections plus differentiable perturbation updates.","core_discovery":"The central claim is that the optimal sparse mask can be extrapolated from the point-wise importance of a dense perturbation. From a pre-optimized dense perturbation δ, the authors estimate for each pixel the expected attack loss over binary masks that keep that pixel, using Monte Carlo sampling against the current support. They then keep the highest-scoring pixels (or, under a patch kernel, the highest-scoring patches via a max-out procedure), and repeat an optimize-then-trim loop on a logarithmic schedule of L0 bounds. The resulting attacks, PGDTrim and PGDTrimKernel, achieve higher adversarial success rates than SparseFool, PGD_L0, GreedyFool, and Homotopy across all tested models and L0 budgets on ImageNet, reaching 100 percent success at L0 = 128 for InceptionV3 and standard ResNet50, and at L0 = 224 for a robust ResNet50.","pith_inferences":["The top-k trim could be interpreted as a greedy step for a possibly submodular set function, suggesting a provable approximation guarantee when the loss is near-submodular over selected pixels.","The Monte Carlo estimator in Eq. (6) is a candidate for variance reduction (control variates or stratified sampling), which could cut the 1000-sample cost and make the attack practical for video or real-time settings.","One could invert PGDTrim's point-wise ranking to build saliency maps of the minimal pixel subsets a classifier relies on, giving an interpretability tool rather than only an attack.","The method is evaluated only on single-image attacks; running the trimming loop over a batch of images would produce a universal sparse patch, and it is an open question whether the gains persist in that setting."],"forward_implications":["Attackers gain a practical way to place multiple arbitrary-shaped adversarial patches in one optimization, which prior patch attacks did not support directly.","Defenses can no longer assume sparse-attack evaluations are complete without this baseline, since it outperforms existing sparse attacks on standard, robust, and transformer models.","Because the trimming step does not require differentiability over locations, the method transfers to settings like physical-world patch placement where location optimization is not differentiable.","Reported 100 percent success rates at low L0 budgets indicate that current ImageNet classifiers remain vulnerable to sparse modifications, and robustness evaluations should include such budgets."],"supporting_citations":[{"why":"provides the PGD optimization scheme that the proposed Dropout-PGD procedure and trimming schedule build on.","marker":"Madry et al. [2018]"},{"why":"introduces SparseFool, a sparse attack baseline the paper compares against on ImageNet.","marker":"Modas et al. [2019]"},{"why":"introduces the PGD_L0 binary-optimization attack used as a baseline.","marker":"Croce and Hein [2019]"},{"why":"the paper attributes the GreedyFool sparse attack baseline to this citation and compares against it.","marker":"Goodfellow et al. [2020]"},{"why":"introduces the Homotopy attack baseline compared in the experiments.","marker":"Zhu et al. [2021]"},{"why":"provides the ImageNet dataset used for all evaluations and L0 budget settings.","marker":"Deng et al. [2009]"},{"why":"supplies the adversarially robust ResNet50 model where the method shows its largest improvement.","marker":"Salman et al. [2020]"}],"fun_headline_variants":["Sparse patch attacks improved by point-wise trimming of dense perturbations","Jointly optimizing patch locations and values beats prior sparse attacks","Point-wise trimming extrapolation yields superior sparse adversarial attacks","Sparse attacks via dense trimming surpass prior methods on ImageNet"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire method leans on the assumption that a pixel's importance computed from a dense perturbation stays valid as the perturbation is made sparse, so pixels that are only useful in correlated groups are never discarded by the trimming.","fun_headline_variants_meta":{"raw":{"variants":["Sparse patch attacks improved by point-wise trimming of dense perturbations","Jointly optimizing patch locations and values beats prior sparse attacks","Point-wise trimming extrapolation yields superior sparse adversarial attacks","Sparse attacks via dense trimming surpass prior methods on ImageNet"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000449,"raw_usage":{"total_tokens":2241,"prompt_tokens":896,"completion_tokens":1345,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":1289}},"tokens_in":512,"tokens_out":1345,"duration_ms":10428,"temperature":1.0,"reasoning_tokens":1289,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T13:28:26.228839+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a trained ImageNet model, compute a PGDTrim mask for a given L0 budget, then replace an equal number of the lowest-ranked kept pixels with an equal-size set of medium-ranked pixels chosen for high pairwise correlation with the surviving set, keeping perturbation magnitudes fixed; if the replacement achieves equal or higher adversarial success, the point-wise independence assumption behind the ranking is untenable.","supporting_citations":[{"cited_title":"Sparsefool: a few pixels make a big difference","cited_arxiv_id":null,"evidence_quote":"introduces SparseFool, a sparse attack baseline the paper compares against on ImageNet."},{"cited_title":"Sparse and imperceivable adversarial attacks","cited_arxiv_id":null,"evidence_quote":"introduces the PGD_L0 binary-optimization attack used as a baseline."},{"cited_title":"Sparse and imperceptible adversarial attack via a homotopy algorithm","cited_arxiv_id":null,"evidence_quote":"introduces the Homotopy attack baseline compared in the experiments."},{"cited_title":"Do adversarially robust imagenet models transfer better? Advances in Neural Information Processing Systems, 33: 0 3533--3545, 2020","cited_arxiv_id":null,"evidence_quote":"supplies the adversarially robust ResNet50 model where the method shows its largest improvement."}],"review_version":1}