{"id":"1689a61f-f4ae-443f-a177-422f6b7795bf","arxiv_id":"2602.05175","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A classifier trained with SDF-based shape guidance and stochastic appearance debiasing is claimed to reach 81.64% robust accuracy under AutoAttack on ImageNet, but the evaluation protocol inflates the result.","lead":"This paper proposes ShapePuri, a training-time defense that uses Signed Distance Fields computed from image masks plus stochastic appearance transforms to make a classifier robust to adversarial attacks, claiming 84.06% clean and 81.64% AutoAttack robust accuracy on ImageNet. The claim is undermined by an evaluation that trains on AutoAttack and compares models of different size on a non-standard training split.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline 81.64% AutoAttack accuracy is in-sample: training and evaluation use the same AutoAttack ensemble, and the SOTA comparison is confounded by architecture mismatch. A held-out attack evaluation is needed.","rationale":"The reader's weakest_assumption is exactly the load-bearing concern: the evaluation is circular because the same AutoAttack ensemble is used for both training and evaluation. This directly undermines the headline claim of a generalizable 80%+ robust accuracy. The method itself is internally coherent—the SDF/GAD ablations (Tables 2–4) show consistent improvements over alternative components—but internal consistency does not validate the external robustness claim. The architecture mismatch in Table 1 further weakens the SOTA comparison, and the paper's own ResNet-50 result (73.68%) is below the OSCP baseline (74.19%), indicating that the headline number is tied to the larger ConvNeXt-L model, not to the proposed method per se. These issues are sufficient to reject the central claim as presented. Since the reader already reached REJECT and my analysis supports that conclusion, the verdict should remain unchanged.","tokens_in":10736,"tokens_out":6211,"duration_ms":66648,"concrete_test":"Obtain the trained ConvNeXt-L checkpoint (or retrain if unavailable) and evaluate it under a held-out attack family not used in training: PGD-100 with 10 random restarts at epsilon=4/255, step=1/255, on the same 10k test images. If robust accuracy drops by more than 5 percentage points relative to the reported 81.64%, the AutoAttack result is in-sample and the 'first >80%' claim does not generalize. Ideally also report the same evaluation for a PGD-100-trained ShapePuri model to isolate the effect of training on AutoAttack.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—first defense to surpass 80% AutoAttack accuracy on ImageNet—rests on an evaluation protocol that makes the result in-sample. Section 4.1 states: 'We adopt the same attack families for training and evaluation to ensure consistency' and 'In all experiments, the attack models are same to the training models.' Thus the reported 81.64% on ConvNeXt-L is measured on the exact AutoAttack ensemble (APGD-CE, APGD-DLR, FAB, Square) used to generate adversarial training examples. A classifier optimized against a specific attack ensemble can score high on that ensemble without generalizing to unseen attacks; the number does not establish robustness as a general property. The SOTA comparison is additionally confounded: the 81.64% is obtained with ConvNeXt-L, whereas the diffusion-based baselines (OSCP, DiffPure) are on ResNet-50. Table 5 shows ShapePuri on ResNet-50 under AutoAttack gives 73.68%, below the cited OSCP baseline of 74.19%. The paper provides no evaluation on a held-out attack family, no matched-architecture baseline, and no code/checkpoint, so the 'unprecedented' margin could be an artifact of training on the evaluation attack plus using a larger model.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes ShapePuri, a training-time defense that combines a Signed Distance Function (SDF) shape encoding module (SEM) and a stochastic Global Appearance De-biasing (GAD) module. During training the classifier is supervised on five input streams: clean, adversarial, SDF-fused adversarial, and GAD-transformed clean and adversarial images. At inference the auxiliary modules are discarded (§3.4, Algorithm 1). On a 40k/10k split of the ImageNet validation set (§4.1), the paper claims 84.06% clean and 81.64% AutoAttack robust accuracy with ConvNeXt-L, and presents this as \"the first defense framework to surpass the 80% threshold on ImageNet\" (§4.2). The paper is essentially an empirical robustness study; no theory, code, or checkpoints are provided.","tokens_in":10993,"tokens_out":8282,"duration_ms":90576,"significance":"The underlying idea—using dense distance fields as structural anchors and stochastic appearance augmentation to reduce texture bias—is potentially interesting, and the deterministic-inference property is practically attractive. If the empirical claims were established, this would be a notable contribution. However, the claims are not established: the evaluation uses the same attack family for training and testing, the headline comparison is confounded by architecture, and the provided ablations do not supply a matched-protocol baseline. The manuscript also contains multiple internal inconsistencies (title/abstract/numbers, module names). I therefore cannot regard the central result as supported.","major_comments":[{"comment":"The evaluation protocol is in-sample. Section 4.1 states \"We adopt the same attack families for training and evaluation to ensure consistency\" and \"In all experiments, the attack models are same to the training models.\" Thus the reported 81.64% AutoAttack accuracy is measured on the exact AutoAttack ensemble (APGD-CE, APGD-DLR, FAB, Square) used to generate training examples. No held-out attack family, loss function, perturbation budget, or adaptive attack is evaluated; Table 6 varies only the attack architecture, not the attack algorithm. The headline \"unprecedented 81.64%\" is therefore an in-sample fit to a specific attack ensemble, not evidence of general adversarial robustness.","section":"§4.1, Tables 1 and 6"},{"comment":"The state-of-the-art comparison is confounded by architecture. ShapePuri's 81.64% AutoAttack result is achieved with ConvNeXt-L, while the diffusion-based baselines (DiffPure, OSCP) use ResNet-50. Table 5 shows ShapePuri with ResNet-50 under AutoAttack at 73.68%, below OSCP's 74.19%. The 7.45% margin claimed in §4.2 therefore cannot be attributed to the method without a matched-architecture comparison. In addition, training on a 40k-image split of the validation set (§4.1) is not shown to be comparable to the baselines' training protocols; this is another source of mismatch.","section":"Table 1 vs. Table 5"},{"comment":"The SDF definition is internally inconsistent. Eq. (1) defines φ(x) but the right-hand side uses I rather than x, and assigns positive distances inside Ω. Eqs. (2)–(3) compute I_SDF = DT(I_mask) − DT(1 − I_mask), which is typically negative inside the foreground under the standard distance-transform convention. The sign convention and variable dependence must be corrected for the method to be reproducible. The foreground-consistency refinement (τ, δ, flood-fill) is also underspecified.","section":"§3.1, Eqs. (1)–(3)"},{"comment":"Reproducibility and ablation reporting are incomplete. Hyperparameters β, τ, δ, the Gaussian blur σ, and the GAD first-layer channel count are not reported; no error bars or multiple seeds are given; and no code or checkpoints are provided. Table 5 reports a ConvNeXt-L targeted-attack accuracy of 82.87/81.36, which is not the number used in the main Table 1 targeted row (80.73/79.02 on ResNet-152), and Table 2's prose refers to \"GIR\" although the module is named GAD. These omissions make it impossible to verify the claimed margin.","section":"§4.3, Tables 2, 4, 5"}],"minor_comments":[{"comment":"The front matter gives a different title, method name (GeoTexPuri), and numbers (84.79% clean / 83.52% robust) from the full text (ShapePuri, 84.06% / 81.64%). This inconsistency must be harmonized.","section":"Title/abstract vs. full text"},{"comment":"The geometric score S_k = A_k \\bar{D}_k^2 is introduced without justification and its units are unclear. Please define the rationale or remove it.","section":"Eq. (4)"},{"comment":"The alternative shape encodings (Canny edges, contours, skeletons) are not given with their own hyperparameters (e.g., Canny thresholds), so the comparison in Table 3 is not fully reproducible.","section":"§4.3, Table 3"},{"comment":"The five-stream diagram is hard to parse at the current resolution. Annotating each stream with its corresponding loss term would improve clarity.","section":"Fig. 2"}],"recommendation":"reject","confidential_remarks":"The appended \"SKEPTIC\" note and the axiom ledger, if they are part of the submitted material, explicitly acknowledge the in-sample evaluation and the architecture confound. My reading of Section 4.1 and Tables 1/5 independently leads to the same conclusion. The central claim would require re-running the experiments with a held-out attack family and matched architectures, which is beyond a revision of the current manuscript."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea is worth a look: using SDFs as dense geometric guidance and a stochastic appearance-debiasing module inside a multi-stream adversarial training objective is a legitimate new combination of known components. The inference-time story is attractive—deterministic, zero overhead, no diffusion sampling. The intuition that shape bias helps robustness is well-grounded in prior work, and the ablations on SDF vs. edges/contours/skeletons show the authors are thinking about what the shape prior should be.\n\nBut the empirical claim does not hold as stated. Section 4.1 says they use the same attack families for training and evaluation, and that the attack models are identical to the training models. So the 81.64% AutoAttack number is measured on the exact ensemble they trained against. That is an in-sample result; it says nothing about robustness to unseen attacks. There is no held-out attack, no adaptive attack, no cross-family evaluation. Table 6 only varies the attack architecture within the same AutoAttack loss family, which does not break the circularity.\n\nThe SOTA comparison is also confounded. The headline 81.64% comes from ConvNeXt-L, while the diffusion baselines are on ResNet-50. The paper's own ResNet-50 AutoAttack result is 73.68%, below OSCP's 74.19%. So the 'first to surpass 80%' claim is an artifact of using a bigger backbone. Clean accuracy also jumps by 6% relative to baselines, which is suspicious if the protocol is otherwise identical. No error bars, no code or checkpoints, no reproducibility artifacts—this cannot be evaluated as a benchmark contribution.\n\nThere are also internal inconsistencies that should have been caught before submission. The abstract in the metadata describes a different framework, GeoTexPuri, with different numbers (84.79/83.52), while the full text is ShapePuri with 84.06/81.64. The ablation Table 2 uses a term 'GIR' that is never defined in the method or figure. These are not fatal to the underlying idea, but they signal the draft is not ready.\n\nThe citation pattern is fine—the references are relevant and not self-promotional. The method itself is coherent, and the problem being addressed (diffusion purification is slow and lossy) is real. But as presented, the paper overclaims. The right move in peer review is to send it back for major revision: retrain with a clean attack protocol, report matched-architecture baselines, add held-out attacks, and release code. I would not cite the current numbers, but I would engage with the idea if it were supported by an honest evaluation.","headline":"The headline 81.64% AutoAttack number is in-sample: they train and evaluate on the same attack ensemble, and the SOTA comparison uses a bigger architecture, so the central claim rests on a circular protocol rather than demonstrated robustness.","tokens_in":796,"tokens_out":1056,"would_cite":false,"duration_ms":26611,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A train-time defense using signed distance fields achieves 81.64% AutoAttack robustness on ImageNet, first to pass 80%.","keywords":["adversarial robustness","adversarial purification","signed distance fields","geometric invariance","appearance debiasing","AutoAttack","ImageNet","shape bias"],"falsifier":"Run the final ConvNeXt-L model against a different attack family not present in training — for example PGD-100 with a ResNet-50 surrogate, a transfer attack from a held-out model, or an adaptive attack that backpropagates through the SDF fusion step — and compare robust accuracy. If accuracy drops steeply (e.g., below 60%), the AutoAttack result is likely due to training on the same ensemble. Also check whether clean accuracy on the full ImageNet train/test split matches the 84.06% reported on the 10k validation split.","tokens_in":10518,"feed_emoji":"🛡️","tokens_out":5244,"duration_ms":49408,"temperature":0.7,"pith_summary":"ShapePuri claims that a deterministic, train-time-only purification framework can make an ImageNet classifier robust to adversarial attacks by anchoring it to geometric invariants rather than appearance. The framework fuses adversarial images with a signed distance field of the object shape and, in parallel, trains the classifier on stochastically appearance-debiased versions of both clean and adversarial images. Under the AutoAttack benchmark, the method reports 81.64% robust accuracy on ImageNet with a ConvNeXt-L backbone, a 7.45% gain over the previous diffusion-based state of the art, and claims to be the first defense to exceed 80% robust accuracy. If these numbers hold, the result would show that expensive diffusion purification is unnecessary for high robustness, since the model itself can internalize structure-consistent representations at training time and run as a standard classifier at inference.","feed_headline":"Shape anchors push ImageNet defense past 80% AutoAttack","feed_subtitle":"A train-time-only method using signed distance fields and appearance debiasing beats diffusion purification with zero inference cost.","key_machinery":"The key mechanism is the signed distance function (SDF) — a continuous scalar field representing the signed Euclidean distance from each pixel to the nearest object boundary, computed from a binary mask via inner and outer distance transforms. Because adversarial perturbations alter pixel intensities but leave the geometric skeleton intact, the SDF provides a stable structural anchor. The framework fuses it with adversarial inputs as I_fusion = I_adv ⊙ (1 + β·I_SDF), amplifying interior regions and attenuating background. The supporting mechanism is Global Appearance Debiasing (GAD), a family of shallow stochastic convolutional networks with weights sampled from N(0,1), whose outputs are int","core_discovery":"The central claim is that training a classifier with a multi-stream objective that includes (1) a shape-guided stream where adversarial images are multiplied by (1 + β·SDF) for β=0.5, and (2) an appearance-debiased stream where images are passed through a random two-layer convolutional network with Gaussian weights, forces the model to rely on stable geometric structure rather than fragile texture. The paper reports that this yields 84.06% clean accuracy and 81.64% robust accuracy against AutoAttack on ImageNet, surpassing all previous defenses and, for the first time, crossing the 80% robust-accuracy threshold. The method requires no auxiliary modules or extra compute at inference; the clas","pith_inferences":["The manuscript's abstract reports different headline numbers (84.79% clean, 83.52% robust) than the body and main table (84.06% clean, 81.64% robust); the paper should be read with this internal inconsistency in mind.","Because the same AutoAttack family is used for both training and evaluation, the robustness claim may not transfer to unseen attack families; an independent evaluation against PGD, momentum-based attacks, or adaptive attacks that differentiate through the SDF fusion would test generalization.","The training split uses 40,000 images from the ImageNet validation set, not the standard 1.2M-image training set; scaling to full data or testing on the official test set could change the clean/robust balance.","The SDF's mask-refinement heuristic (inverting the mask when the mean intensity exceeds threshold τ) assumes background dominance; for close-up or large-object images the geometric anchor may be less reliable, which might limit performance on fine-grained classes."],"forward_implications":["If the result is correct, robust inference no longer requires generative purification; a standard classifier trained this way is as fast as an undefended model.","The 80% AutoAttack milestone suggests robust accuracy on ImageNet is not bounded far below clean accuracy; a gap of only ~2.4% (84.06 clean vs 81.64 robust) is claimed.","The approach is architecture-agnostic: the paper reports gains on ResNet variants, ConvNeXt-L, and Swin-B, suggesting geometric anchoring transfers across backbones.","The use of SDF as a dense prior could extend to other tasks where object boundaries are meaningful, such as medical image segmentation or video object tracking, though the paper does not explore these."],"fun_headline_variants":["Zero-cost defense: SDF-guided training beats diffusion purification","Train-time SDF anchors push robust accuracy past 80%","No inference overhead: shape-texture balancing hits 80%+ robust","SDF training harmonizes geometry and texture for robust defense","Deterministic classifier with zero extra compute: robust to AutoAttack"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The paper's headline robustness numbers are measured under the same AutoAttack attack ensemble that was used to generate training adversarial examples (Section 4.1), so the reported 81.64% may reflect overfitting to that specific attack set rather than generalizable robustness.","fun_headline_variants_meta":{"raw":{"variants":["Zero-cost defense: SDF-guided training beats diffusion purification","Train-time SDF anchors push robust accuracy past 80%","No inference overhead: shape-texture balancing hits 80%+ robust","SDF training harmonizes geometry and texture for robust defense","Deterministic classifier with zero extra compute: robust to AutoAttack"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00097,"raw_usage":{"total_tokens":3966,"prompt_tokens":750,"completion_tokens":3216,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":494,"completion_tokens_details":{"reasoning_tokens":3139}},"tokens_in":494,"tokens_out":3216,"duration_ms":24891,"temperature":1.0,"reasoning_tokens":3139,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T04:18:48.447019+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the final ConvNeXt-L model against a different attack family not present in training — for example PGD-100 with a ResNet-50 surrogate, a transfer attack from a held-out model, or an adaptive attack that backpropagates through the SDF fusion step — and compare robust accuracy. If accuracy drops steeply (e.g., below 60%), the AutoAttack result is likely due to training on the same ensemble. Also check whether clean accuracy on the full ImageNet train/test split matches the 84.06% reported on the 10k validation split.","supporting_citations":[],"review_version":1}