{"id":"08705fc7-a08a-4248-8ef4-969e60b848a6","arxiv_id":"1908.01469","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":2,"one_line_summary":"Smoothing inputs with a low-pass filter and comparing the classifier's confidence before and after smoothing detects FGSM and PGD adversarial examples on MNIST and ImageNet.","lead":"This paper proposes an adversarial example detector that smooths each input with a Gaussian or median filter, then checks whether the classifier's confidence in its top class drops. On MNIST and ImageNet images attacked with FGSM and PGD, it reports near-perfect detection rates, but thresholds are tuned on the test set and a promised proof is missing.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"High-frequency assumption is asserted, not proven, and is untested against adaptive or low-frequency attacks, so the claimed 99.7–100% detection rates do not support the central claim.","rationale":"The paper's central claim is that low-pass filtering followed by confidence comparison detects adversarial examples at 99.7–100% in an automated, end-to-end manner. The entire mechanism depends on the assertion that adversarial perturbations are high-frequency, so the filter removes them while leaving benign images intact. This assumption is explicitly stated in Sec. 1 and Sec. 3.2, but the promised 'theoretical proof' does not appear in Sec. 3; the only evidence is one worked example (Fig. 3). Moreover, the experiments only test FGSM and PGD attacks generated without knowledge of the detector. An adversary who is aware of the sieve can design perturbations that survive low-pass filtering, either by using low-frequency energy or by directly optimizing the perturbation so that the filtered confidence change stays below the detection threshold. Such adaptive attacks are the standard robustness requirement for a claimed detection system, and their absence makes the reported detection rates insufficient support for the central claim. The threshold values also appear to be chosen on the test data, which further weakens the empirical claim. Since the load-bearing concern is the unproven and untested high-frequency assumption, and the reader's weakest_assumption identified the same issue, I agree with the reader's rejection; no verdict change is needed.","tokens_in":9621,"tokens_out":4202,"duration_ms":42096,"concrete_test":"Implement an adaptive PGD attack against the full detector pipeline: given a benign image x with class y_true and target y*, optimize δ (with L∞ bound) to minimize L_class(f(x+δ), y*) + λ * max(0, |p_{y*}(f(G(x+δ)) - p_{y*}(f(x))| - Θ), where G is the Gaussian low-pass filter used by DSG and Θ is the detection threshold, so the filtered confidence stays close to the original. Then run the detector on 1,000 such adaptive examples and 1,000 benign images. If the detection rate drops below the claimed 99.7% (e.g., to chance), the high-frequency assumption is insufficient and the central claim fails. Alternatively, generate low-frequency adversarial perturbations by adding only smooth (e.g., Gaussian-blurred) noise to the input and test the detector.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The detection mechanism rests on the premise that adversarial perturbations are high-frequency features (Sec. 1 and Sec. 3.2), so low-pass filtering removes the attack while preserving benign confidence. The paper promises a proof in Sec. 3 but provides none; the only support is a single qualitative example (Fig. 3). The evaluation considers only non-adaptive FGSM and PGD attacks on two datasets; no adversary is assumed to know the sieve. Because the detector's threshold compares classifier confidence before and after fixed Gaussian/median filtering, an adaptive adversary can craft perturbations that (a) change the classification to a target class, and (b) remain nearly unchanged after low-pass filtering, e.g., by concentrating perturbation energy in low spatial frequencies. If such attacks evade detection, the central claim that the system automatically distinguishes adversarial from benign inputs in general settings is false. The different thresholds used per dataset (0.1 for MNIST, 0.92 for ImageNet) are also selected post hoc, and 'fixed' is only within each dataset.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an automated detection system for adversarial examples based on the hypothesis that adversarial perturbations are predominantly high-frequency. The system runs the input through a low-pass filter (Gaussian or median) and compares the classifier's confidence in the original highest-confidence class before and after filtering; if the difference exceeds a threshold, the input is flagged as adversarial. The authors evaluate on MNIST and ImageNet with FGSM and PGD attacks and report detection accuracies of 99.7–100% in many settings, claiming superiority over feature squeezing (Xu et al.). Section 3 is said to contain a theoretical proof of the frequency-domain hypothesis, and the paper claims the system is fully automatic with a fixed threshold.","tokens_in":9846,"tokens_out":5496,"duration_ms":52596,"significance":"If the claimed detection rates were valid, the work would be a useful contribution to adversarial-example detection, and the algorithm-style presentation (Algorithms 1 and 2) makes the method easy to test and compare. The paper also identifies threshold selection as a weakness of prior work. However, the evidence presented is not sufficient to establish the claims: the promised theoretical proof is absent, the thresholds appear to be chosen using the test data, no adaptive attacks are considered, and the evaluation is narrow (two datasets, two attacks, no error bars). The central hypothesis about high-frequency adversarial noise is asserted rather than demonstrated, and the reported detection rates are not reliable as evidence.","major_comments":[{"comment":"The paper states in Section 1 and Section 3 that a theoretical proof of the high-frequency assumption is provided, but no such proof appears anywhere in the text. The only supporting evidence is the qualitative example in Fig. 3. Since the entire detection mechanism rests on the premise that low-pass filtering removes adversarial noise while preserving benign features, this missing proof is a load-bearing gap that must be addressed.","section":"Section 3, especially Sec. 3.2"},{"comment":"The thresholds Θ=0.1 (MNIST) and Θ=0.92 (ImageNet) are reported as 'fixed,' but the manuscript does not describe how they were chosen or whether they were selected on a validation set disjoint from the test set. Reporting 99.7–100% detection on the same data used to determine the thresholds is circular and likely overstates performance. The paper should provide a threshold-selection procedure, error bars, and results on held-out sets.","section":"Tables 1 and 2"},{"comment":"The evaluation considers only white-box FGSM and PGD attacks; no adversary that knows the sieve mechanism is tested. Because the detection rule is deterministic (comparing filtered and unfiltered classifier confidence), an adaptive adversary can craft perturbations with low-frequency energy or perturbations that keep the filtered confidence nearly unchanged. Without such experiments, the claim that the system can 'mostly distinguish adversarial samples and benign images in an end-to-end manner' is unsupported.","section":"Section 4.3 and Algorithm 2"},{"comment":"The comparison with Xu et al. [21] is not controlled: the datasets differ in size (3,000 vs 1,800 for ImageNet), class balance, and threshold choices, and no confidence intervals are reported. The statement that 'our detection rates exceeded those of Xu et al.' is therefore not justified by the data presented.","section":"Section 4.1 and Table 2"},{"comment":"The sentence 'The proposed detection system knows the true labels of the input' directly contradicts the claimed automatic, end-to-end operation. If the true label is used anywhere in the detection decision, the method is not an automated detector; if not, the sentence must be corrected. This ambiguity undermines the paper's central claim.","section":"Section 4.2"}],"minor_comments":[{"comment":"The heading 'Related W orks' contains a typo and should read 'Related Works'.","section":"Section 2.1 heading"},{"comment":"'Carnili et al.' should be 'Carlini et al.'; the reference list correctly cites Carlini and Wagner [2].","section":"Section 2.1"},{"comment":"Equation (3) is malformed: the projection operator 'project (x,ϵ)(x∗)' is not clearly defined. In Algorithm 1, the first line 'x← x∗' should be 'x∗← x' to initialize the adversarial sample from the clean input.","section":"Equation (3) and Algorithm 1"},{"comment":"The abstract and conclusion overstate the results: the abstract reports an out-detection rate of 99.7–100%, but Table 2 shows accuracy of 0.983 (DSG) and 0.958 (DSM) on ImageNet, with only DSM recall reaching 1.0. Section 5 claims 'maximum accuracy rates of 99.9% and 100%,' which does not match the accuracy values in Tables 1 and 2.","section":"Abstract, Section 4.3, and Section 5"},{"comment":"The caption contains spelling errors ('Orginal', 'suffer to DSG') and should be corrected for clarity.","section":"Figure 3 caption"},{"comment":"Algorithm 2 loops over kernel sizes κ = [(3×3); (5×5)], but the final decision uses p(sievey) without specifying how results from different kernel sizes are aggregated; the effect of κ on the output should be clarified.","section":"Algorithm 2"}],"recommendation":"reject","confidential_remarks":"The manuscript appears to be an early draft with several fundamental issues: the claimed theoretical proof is absent, threshold selection appears circular, and the evaluation lacks any adaptive adversary. These are not merely presentation problems; they affect the validity of the reported detection rates. If the authors can provide a proper threshold selection procedure, test on held-out data, and evaluate against adaptive attacks, a future resubmission might be considered, but the current paper does not meet the standard for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper is a thin variant of feature squeezing with an extra fixed-threshold anchor comparison. The headline numbers are not believable as stated because the promised proof in Section 3 is absent, and the thresholds are chosen after seeing the test set. The high-frequency assumption—load-bearing—is asserted, not demonstrated.\n\nWhat's actually useful: the method is simple and easy to reproduce. The authors cite Xu et al. directly, and they are upfront that the sieve idea is a low-pass filter before the classifier. The one concrete contribution is the anchor-based threshold rule, which might be a small improvement in usability over feature squeezing's per-case thresholds. On FGSM and PGD white-box attacks on MNIST and ImageNet, the high recall numbers are plausible, and the single worked example in Fig. 3 is consistent with the mechanism.\n\nBut the paper has three soft spots that matter. First, Section 3 promises a theoretical proof of the high-frequency assumption and the detector's correctness; none is delivered. The text just restates the assumption and shows a picture. That is a serious omission, not a style issue. Second, the thresholds (0.1 on MNIST, 0.92 on ImageNet) are called 'fixed' but there is no explanation of how they were set. Given the numbers, they look fit to the test set. The paper even says 'the detection system knows the true labels of the input' in Section 4.2, which suggests the evaluation is not a proper blind test. Third, the threat model is limited to non-adaptive FGSM/PGD. The whole defense relies on attacks living in high frequencies; an adaptive adversary or a low-frequency attack would break it. No such test is reported.\n\nThe comparison to Xu et al. is also sloppy: they use imbalanced test sets, different numbers of files, and different attack parameters, and then claim superiority. The F1 numbers are not apples-to-apples.\n\nBottom line: the idea is fine as a baseline, and the paper is readable. But it overclaims, and the missing proof plus the threshold-fitting issue mean the central claim is not supported. I would not send this to peer review in its current form—it needs either a real proof or a much better evaluation with adaptive attacks and fixed a-priori thresholds. It could be a workshop poster or a short technical report, but not a full paper.","headline":"Thin variant of feature squeezing with a missing proof and thresholds fit to the test set; the 99.7–100% claim is not supported.","tokens_in":10324,"tokens_out":2704,"would_cite":false,"duration_ms":26475,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A low-pass filter sieve can distinguish adversarial images from benign ones with 99.7–100% accuracy in many settings.","keywords":["adversarial examples","deep neural networks","high-frequency noise","low-pass filter","Gaussian filter","median filter","FGSM","PGD"],"falsifier":"Craft adversarial examples whose perturbation energy is concentrated in low-frequency bands, or optimize FGSM/PGD perturbations against the sieve itself so the anchor–filtered probability difference stays below the fixed threshold while the classifier still misclassifies; if such examples succeed on MNIST or ImageNet, the high-frequency premise is false.","tokens_in":9455,"feed_emoji":"🔍","tokens_out":7581,"duration_ms":68949,"temperature":0.7,"pith_summary":"The paper argues that adversarial perturbations, at least for the FGSM and PGD attacks it studies, are concentrated in the high-frequency content of an image. It proposes an end-to-end detector that feeds the input to a classifier twice: once directly to record the highest-confidence anchor class, and once through a Gaussian or median low-pass 'sieve' that strips high-frequency noise. If the filter changes the anchor class's probability by more than a fixed threshold, the input is labelled adversarial; otherwise it is benign. On MNIST and ImageNet test sets built by the authors, the system reports detection rates of 99.7–100% in many settings and can sometimes recover the original class from an adversarial image. A sympathetic reading is that a retraining-free, threshold-based pre-filter can neutralize common gradient-based attacks.","feed_headline":"Sieve filter catches 99.7–100% of adversarial images","feed_subtitle":"Gaussian and median filtering expose attack noise by changing only adversarial confidence scores, with no retraining.","key_machinery":"The carrying object is the 'sieve' layer: either a Gaussian low-pass filter $G_\\sigma(i,j) = \\frac{1}{2\\pi\\sigma^2} e^{-(i^2+j^2)/(2\\sigma^2)}$ (Detection System based on Gaussian, DSG) or a median filter (Detection System based on Median, DSM), applied with $3\\times 3$ and $5\\times 5$ kernels. The system runs input through two parallel flows: the anchor flow classifies the raw input and records the highest-confidence class, and the sieve flow filters the input and then classifies it conditional on that anchor. A fixed threshold $\\Theta$ on the probability difference turns the filter into a binary detector; no retraining or per-dataset threshold search is required.","core_discovery":"The central claim is that a low-pass filter placed before a classifier exposes adversarial examples: when the input is adversarial, filtering removes the perturbation and makes the model's confidence in its original top class collapse, while for a benign image the same filtering barely moves the top-class probability. The proposed system formalizes this as a sieve in parallel with an anchor: the direct input path supplies the anchor class, the sieved path is classified given that anchor, and the difference between the two probabilities is compared with a fixed threshold $\\Theta$. The paper states that the high-frequency assumption is supported by a theoretical proof, and the experimental sections support it empirically with FGSM and PGD attacks on MNIST and ImageNet.","pith_inferences":["The paper leaves implicit that an adversary aware of the sieve could shift perturbation energy into low-frequency bands or optimize the perturbation to keep the filtered anchor probability stable; testing such adaptive attacks would directly probe the high-frequency premise.","The same anchor-versus-filtered comparison generalizes to other denoisers, such as bilateral filters or wavelet shrinkage, because the mechanism only requires that attack noise be attenuated more than benign content.","The reported 99.7–100% figures are specific to the constructed test sets and attack settings; worst-case guarantees would require a different evaluation, so the numbers should be read as strong evidence for the mechanism rather than as an upper bound on detection difficulty."],"forward_implications":["A deployment can add adversarial detection without retraining or modifying the target classifier: only a Gaussian or median filter and a fixed threshold are inserted in front of it.","Because the decision requires only the classifier's confidence outputs, the same detector design transfers across architectures and datasets, as shown with MNIST and ImageNet.","The filtered output can be used not just to reject adversarial inputs but, in many cases, to recover the true class from an adversarial image, effectively a preprocessing defense as well as a detector.","Compared with feature-squeezing detectors that need per-setting thresholds, the fixed-threshold design removes a manual tuning step that could overburden operators."],"supporting_citations":[{"why":"Supplies the FGSM attack used to generate adversarial examples.","marker":"[3]"},{"why":"Supplies the PGD attack used to generate adversarial examples and the adversarial-training baseline.","marker":"[12]"},{"why":"The feature-squeezing detector the paper compares against, whose per-setting threshold tuning motivates the fixed-threshold design.","marker":"[21]"},{"why":"Provides the MNIST benchmark used for the small-scale detection evaluation.","marker":"[10]"},{"why":"Provides the ImageNet benchmark used for the large-scale evaluation.","marker":"[16]"},{"why":"Defines the Inception V3 model used as the ImageNet classifier.","marker":"[18]"},{"why":"A prior denoising defense (HGD) that motivates the filtering-based approach and is contrasted for its lack of testing on grayscale datasets.","marker":"[11]"}],"fun_headline_variants":["Frequency sieve detects adversarial images 99.7-100%","Low-pass filter exposes adversarial attacks without retraining","Sieve method catches adversarial examples at 99.7-100% rate","Adversarial detection via high-frequency sieve with fixed threshold"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Everything hangs on the claim that adversarial perturbations are high-frequency noise, a premise the paper states as proven in Section 3 without actually presenting that proof.","fun_headline_variants_meta":{"raw":{"variants":["Frequency sieve detects adversarial images 99.7-100%","Low-pass filter exposes adversarial attacks without retraining","Sieve method catches adversarial examples at 99.7-100% rate","Adversarial detection via high-frequency sieve with fixed threshold"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000287,"raw_usage":{"total_tokens":1617,"prompt_tokens":810,"completion_tokens":807,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":426,"completion_tokens_details":{"reasoning_tokens":737}},"tokens_in":426,"tokens_out":807,"duration_ms":7758,"temperature":1.0,"reasoning_tokens":737,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T15:11:07.336650+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Craft adversarial examples whose perturbation energy is concentrated in low-frequency bands, or optimize FGSM/PGD perturbations against the sieve itself so the anchor–filtered probability difference stays below the fixed threshold while the classifier still misclassifies; if such examples succeed on MNIST or ImageNet, the high-frequency premise is false.","supporting_citations":[{"cited_title":"In: 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proceedings (2018)","cited_arxiv_id":null,"evidence_quote":"Supplies the PGD attack used to generate adversarial examples and the adversarial-training baseline."},{"cited_title":"In: 25th Annual Network and Distributed System Security Symposium, NDSS 2018, San Diego, California, USA, February 18-21 (2018)","cited_arxiv_id":null,"evidence_quote":"The feature-squeezing detector the paper compares against, whose per-setting threshold tuning motivates the fixed-threshold design."},{"cited_title":"AT&T Labs [Online]","cited_arxiv_id":null,"evidence_quote":"Provides the MNIST benchmark used for the small-scale detection evaluation."},{"cited_title":"In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition","cited_arxiv_id":null,"evidence_quote":"A prior denoising defense (HGD) that motivates the filtering-based approach and is contrasted for its lack of testing on grayscale datasets."}],"review_version":1}