{"id":"e5a4465a-163b-487e-b74b-b4b2ce1e7ad9","arxiv_id":"2412.16662","paper_version":2,"verdict":"REJECT","confidence":"HIGH","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"The proposed GAN-based adversarial attack reduces, by its own equations, to the standard BIM attack and is evaluated only on MNIST.","lead":"This paper claims to use a generative adversarial network to create tiny image changes that fool image classifiers, with tests on a handwriting dataset. The actual method section gives the standard BIM gradient attack, and the promised multi-dataset evaluations and projection operation do not appear in the experiments.","discovery_kind":"incremental","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed GAN-based attack is, in the paper's own formalization, a verbatim BIM update; no generator or discriminator training objective is connected to the attack, so the central novelty and mechanism are unsupported.","rationale":"The reader's verdict is REJECT, and the weakest assumption identified is that the method is genuinely GAN-based and distinct from standard gradient attacks. My independent reading of the manuscript confirms this concern. The method section contains the standard GAN min-max objective as background (Section III-A), but the actual attack update in Equation (3) is exactly the BIM update from [12], and Algorithm 1 contains no reference to the generator or discriminator. The paper never specifies how adversarial learning between the generator and the classifier is performed, how the discriminator is used (or trained), or how Equations (4)-(5) affect the algorithm or experiments. The experimental section is limited to MNIST with one classifier, and the contributions listed at the end of the introduction (perturbation space, projection operation, multi-dataset transferability) are not addressed in the experiments. These are mechanical observations about the text, not judgments about intent. I did not find any independent support in the manuscript: there is no code, no formal verification, and no ablation separating GAN components from gradient updates. Therefore the central claim is unsupported, and the verdict should remain REJECT. I agree with the reader's weakest assumption; no adjustment to the verdict is needed.","tokens_in":8024,"tokens_out":1722,"duration_ms":17990,"concrete_test":"Re-implement Algorithm 1 exactly as written, omitting every GAN component (generator, discriminator, minimax loss), and run it on MNIST with the paper's stated settings (10 iterations, perturbation size 0.1, step size 0.01) against the same 3-layer CNN. If the resulting adversarial samples and attack success rate are bitwise identical to standard BIM with the same hyperparameters, or statistically indistinguishable, then the method reduces to BIM and the GAN-based claim is vacuous. Also check the paper's code/artifacts (if any) for any use of G or D in the attack loop; absence of such use would confirm the gap.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that a GAN-based adversarial attack outperforms FGSM and BIM while preserving image naturalness (Abstract; Section V). For that claim to hold, the method must actually use a generative adversarial network in a way that distinguishes it from standard gradient-based attacks. The paper's own formalization contradicts this. Section III-C presents Equation (3), x^{(t+1)} = Clip_{x,eps}(x^{(t)} + alpha * sign(grad_x J(theta, x^{(t)}, y))), which is exactly the BIM update from reference [12]. Algorithm 1 then spells out the procedure: forward pass, cross-entropy loss, backpropagation, sign of gradient, update, clipping, repeat. No line references the generator G, the discriminator D, Equation (1)'s minimax objective, or any adversarial training loop tying G, D, and the classifier together. The GAN content in Section III-A is generic background and is not connected to the attack procedure. Equations (4) and (5) introduce a total-variation regularizer and a dynamic weight function, but neither appears in Algorithm 1, in the experimental setup, or in the reported results. Thus the load-bearing premise that 'ours' is a GAN-based attack distinct from BIM is unsupported by the method as written. Moreover, the contributions listed in the introduction (perturbation space properties, projection operation for transferability, multi-dataset evaluations) do not match the experiments in Section IV, which use only MNIST and one 3-layer CNN, with no transferability or projection experiments. The experimental comparisons in Table 2 and Figures 4-5 therefore cannot validate the stated novelty: if the implemented algorithm is BIM with different hyperparameters, the reported 'Ours' results are at best a re-scaled BIM variant, not a GAN-based method. This is an internal inconsistency, not a disagreement with consensus, and it directly undermines the central claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an adversarial attack method against image classifiers, claiming to use generative adversarial networks (GANs) to generate imperceptible perturbations through adversarial learning of a generator and a classifier. The method is compared with FGSM and BIM on the MNIST dataset with a simple three-layer CNN, reporting accuracy drop, attack success rate, computational cost, robustness heatmaps, and feature-space visualizations. The central claim is that the proposed GAN-based attack outperforms FGSM and BIM in success rate and efficiency while preserving image naturalness (Abstract, Section V).","tokens_in":8404,"tokens_out":2898,"duration_ms":26266,"significance":"If the central claim were supported, a GAN-based attack that improves on FGSM and BIM would be a useful contribution to adversarial machine learning, with potential application to robustness evaluation. The paper gives a clear account of the FGSM and BIM baselines and provides a readable description of the GAN background. However, the claimed novelty rests entirely on the assertion that the attack is GAN-based, and the manuscript does not provide a training loop, a generator/discriminator objective tied to the attack, or any experimental evidence that a GAN is actually used. The experimental evaluation is limited to a single simple CNN on MNIST, so the abstract's claim of deceiving 'a variety of advanced classifiers' is unsubstantiated. The paper provides no code, no architectural details, no error bars, and no ablation of the regularizers introduced in Equations (4) and (5). On the evidence in the manuscript, the method reduces to BIM, so the claimed significance does not hold as written.","major_comments":[{"comment":"The proposed update rule in Equation (3) is exactly the BIM update from reference [12], and Algorithm 1 describes only a standard iterative gradient-sign attack: forward pass, cross-entropy loss, backpropagation, sign of gradient, update, clipping. No line of Algorithm 1 uses the generator G, the discriminator D, the minimax objective in Equation (1), or any adversarial training loop tying G, D, and the classifier together. Thus the paper's central claim that the attack is GAN-based and generated through adversarial learning of generator and classifier is unsupported by the method as written.","section":"Section III-C, Eq. (3)"},{"comment":"The abstract and conclusion claim that the method 'successfully deceives a variety of advanced classifiers' and the introduction promises evaluations on 'multiple real-world datasets as well as a synthesized dataset' with transferability across scenarios. The experiments in Section IV use only MNIST and one three-layer CNN, with no transferability experiments, no synthesized dataset, and no advanced classifiers. The claimed contributions in the introduction, including the projection operation for transferability, do not appear in the methodology or experiments. This is a load-bearing mismatch between the claims and the actual evaluation.","section":"Section IV and abstract/conclusion"},{"comment":"The total-variation regularization term in Equation (4) and the dynamic weight function in Equation (5) are introduced as part of the method, but neither appears in Algorithm 1, in the experimental setup, or in the reported results. There is no ablation or any other evidence that these terms affect the attack's performance or the naturalness of the perturbations. Consequently, the statement that the method maintains naturalness through these mechanisms is not supported.","section":"Section III-C, Eqs. (4) and (5)"},{"comment":"Table 2 reports that generating one adversarial sample takes 0.15 seconds for 'Ours' versus 0.47 seconds for BIM, without any description of what distinguishes 'Ours' from BIM computationally. Since the algorithm is the same iterative gradient-sign procedure with the same iteration count and identical operations, this speedup is unexplained and implausible; no hardware, software, or implementation details are given. The absence of error bars or repeated runs makes it impossible to assess whether the reported differences are meaningful.","section":"Section IV-B, Table 2"},{"comment":"The comparison is difficult to interpret because 'Ours' is never defined independently from the baselines; the same hyperparameters (perturbation size 0.1, step size 0.01, 10 iterations) are used, and the method is described only as 'the proposed method' with no distinguishing algorithmic content. The text states that 'the performance of other three methods are extremely unacceptable' when only FGSM and BIM are compared, and no quantitative values for accuracy or attack success rate are given in the text, so the reader cannot verify the claimed superiority from Figures 4 and 5.","section":"Section IV-B, Figures 4-6"}],"minor_comments":[{"comment":"The caption for Figure 2 reads 'Illustration of generative images by using generative adversarial networks,' which appears to be a copy-paste error from Figure 1; the figure is meant to illustrate the principle of adversarial attacks, not generative images.","section":"Figure 2 caption"},{"comment":"The text says 'we compared two different adversarial attack methods: Fast Gradient Sign Method (FGSM) and Basic Iterative Method (BIM)' but then refers to 'these three methods' and 'other three methods'; the wording should consistently identify 'Ours' as the third method, and 'other two methods' when comparing to the baselines.","section":"Section IV-B"},{"comment":"The caption for Figure 7 says 'Feature Space Visualization using t-SNE,' but the surrounding text describes principal component analysis (PCA) for dimensionality reduction; the caption and text should agree on the actual technique used.","section":"Figure 7 caption and text"},{"comment":"The evaluation metrics are enumerated as 'A. Attack Accuracy' and 'B. Adversarial Success Rate' immediately after a sentence about 'the following items,' which is confusing because the letters A and B are also used for other purposes in the paper; using a numbered list would improve clarity.","section":"Section IV-A"},{"comment":"The table heading says 'NOTIONS' and the column says 'Notion symbols'; the intended word is 'Notations.' Also, the table lists parameters such as alpha and beta that are later reused with different meanings in Equations (3) and (5), which is notationally confusing.","section":"Table 1"},{"comment":"The abstract states that 'extensive experiment analysis' was performed, but only one dataset and one classifier are used; the wording should be adjusted to match the actual experimental scope.","section":"Section IV-A"}],"recommendation":"reject","confidential_remarks":"The manuscript does not currently support its central claim that the proposed attack is GAN-based; the formalized algorithm is BIM, and the GAN material is background that is not connected to the attack procedure. The experimental section is far too limited for the claims made in the abstract and introduction, and the computational cost comparison is unexplained. Even with substantial revision, the authors would need to demonstrate a genuine GAN-based training procedure and provide a more comprehensive evaluation, which is beyond the scope of the current manuscript. I recommend rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, this one doesn't hold up. The paper claims a GAN-based adversarial attack, but the only concrete update rule, Eq. (3), is verbatim the BIM update from Kurakin et al. [12]. Algorithm 1 is a standard gradient-descent loop: forward pass, cross-entropy loss, backprop, sign of gradient, clip, repeat. No generator, discriminator, or adversarial training objective appears anywhere in the attack procedure. The GAN content in Section III-A is generic background; Eq. (4) and (5) introduce a total-variation regularizer and a dynamic weighting scheme but neither is used in Algorithm 1 or in the experiments. That is not a missing detail; it is the central mechanism the abstract and conclusion advertise.\n\nWhat is genuinely new: essentially nothing. The listed contributions in the introduction (perturbation-space framework, projection for transferability, multi-dataset evaluation) do not match the experiments, which use only MNIST and a single 3-layer CNN. There is no projection experiment, no transferability test, and no second dataset. The favorable comparisons in Figures 4-6 and Table 2 are uninterpretable because 'Ours' is never defined independently of the baselines. If the implementation is just BIM with a different step size or iteration count, the reported success rates and the claimed 'lowest computation cost' are an artifact of hyperparameter choice, not a new method.\n\nThe paper does a passable job summarizing FGSM, BIM, and some deepfake-protection literature, and the prose is readable. But that is context, not a contribution. There are also small signs of carelessness: Figure 7 is labeled t-SNE while the text describes PCA; the text says 'other three methods' when only three methods are compared; the conclusion repeats that 'advanced classifiers' were deceived, though only one simple CNN was tested.\n\nThis is a load-bearing flaw, not a quibble. The method as written collapses to prior work. I would not send this to peer review in its current form. A serious rewrite would need to specify and train an actual GAN-based attack (including the generator and discriminator update rules), connect the regularizers to the optimization, and evaluate on at least one non-trivial dataset with proper baselines. As it stands, the paper is not a new result and not ready for referee time.","headline":"The central claim is unsupported: the paper's own update rule is standard BIM, with no GAN component in the algorithm or experiments.","tokens_in":8969,"tokens_out":2217,"would_cite":false,"duration_ms":18353,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proposes a GAN-based adversarial attack that generates small, natural-looking perturbations to fool image classifiers, and claims it beats FGSM and BIM in success and cost on MNIST.","keywords":["adversarial attack","image classification","generative adversarial network","adversarial learning","iterative gradient attack","Fast Gradient Sign Method","Basic Iterative Method","MNIST"],"falsifier":"Inspect the training loop and generated samples: if no generator or discriminator is trained, and the reported adversarial samples come from the 10-iteration clipped signed-gradient update of Eq. 3, the GAN-based claim is falsified. A direct check is to run Algorithm 1 with all GAN components removed and see whether the attack success and the 0.15-second cost are unchanged.","tokens_in":7833,"feed_emoji":"🎯","tokens_out":10429,"duration_ms":81979,"temperature":0.7,"pith_summary":"The paper tries to establish that a generative-adversarial-network-based attack can fool image classifiers with small, natural-looking perturbations, and that this attack beats the standard FGSM and BIM baselines. The proposed mechanism is framed as adversarial learning between a generator and the classifier, with an iterative clipped gradient update under a perturbation budget. On MNIST with a three-layer convolutional classifier, the paper reports lower accuracy under attack, higher attack success than FGSM and BIM, natural-looking adversarial samples, and a lower per-sample computational cost of 0.15 seconds, versus 0.25 seconds for FGSM and 0.47 seconds for BIM. A sympathetic reader would care because the method is presented as a proactive way to expose classifier weaknesses before adversaries exploit them.","feed_headline":"GAN attack beats FGSM and BIM on MNIST, paper says","feed_subtitle":"Reported 0.15-second adversarial samples look natural to humans while fooling the classifier.","key_machinery":"The load-bearing object is the iterative update $x^{(t+1)} = \\mathrm{Clip}_{x,\\epsilon}(x^{(t)} + \\alpha \\cdot \\mathrm{sign}(\\nabla_x J(\\theta, x^{(t)}, y)))$ (Eq. 3): repeated clipped sign-gradient steps that maximize the classifier's loss under a perturbation threshold. The paper wraps this update in a GAN formulation (Eq. 1) with a generator and a discriminator, and states that the generator and the classifier are trained adversarially. Two auxiliary mechanisms are added: total-variation regularization (Eq. 4) to suppress noise-like artifacts, and a time-dependent weight function $w(t)$ (Eq. 5) that adaptively adjusts the influence of adversarial samples during training. The same update equation is the one used by the BIM baseline, so it is the mechanism that must distinguish the proposed attack from a standard gradient attack.","core_discovery":"The paper's central claim is that generative adversarial networks can generate adversarial samples that are both effective and natural: small perturbations that change a classifier's decision while remaining visually unnoticeable. The authors state that by adversarially training the generator against the classifier, the attack learns where the classifier is fragile, and they report that on MNIST it outperforms FGSM and BIM in attack success ratio and computational cost. They also claim that the adversarial samples stay close to normal samples in feature space, which they take as evidence of concealment. The paper's stated goal is to reveal weaknesses in image classification systems and to motivate stronger defenses.","pith_inferences":["Inference: Because Eq. 3 and Algorithm 1 match the BIM baseline, removing the GAN framing and rerunning the experiment is the decisive test of whether the proposed mechanism is doing the work.","Inference: The claimed naturalness could be quantified on MNIST by measuring distance to the nearest real-sample neighborhood or by a human-perception study; the paper's PCA and t-SNE plots are suggestive, not a metric.","Inference: If a real generator were trained, the attack might transfer to harder datasets and architectures, which would test the abstract's promise of deceiving a variety of advanced classifiers beyond the one three-layer CNN evaluated."],"forward_implications":["A trained GAN-based attacker could generate adversarial samples without per-image gradient search, lowering the cost per attack.","The same samples could be used to harden classifiers through adversarial training, since the attack exposes the model's fragile regions.","The recipe could be turned into an image-protection tool: applying such perturbations before publishing a photo would make deepfake-style edits fail while the photo still looks normal.","The reported 0.15-second generation time makes the attack fast enough for real-time or large-batch deployment, if the result holds."],"supporting_citations":[{"why":"It defines the FGSM baseline, the single-step gradient attack the proposed method must outperform.","marker":"[11]"},{"why":"It defines the BIM baseline; the paper's Eq. 3 update is the same clipped iterative sign-gradient step.","marker":"[12]"},{"why":"It supplies the generator-discriminator loss that the paper invokes as its GAN basis.","marker":"[2]"},{"why":"It provides prior evidence that adversarial inputs can push pre-trained generators out of their domain, motivating the GAN-based attack.","marker":"[14]"}],"fun_headline_variants":["GAN attack beats FGSM and BIM on MNIST, stays natural","Invisible GAN attacks fool MNIST classifiers in 0.15s","GAN-crafted adversarial images stay natural while fooling classifiers","On MNIST, GAN attack outshines FGSM and BIM, looks real","Adversarial GAN samples: natural and fast, beat FGSM and BIM"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the method is genuinely a GAN-based attack and not just the standard iterative gradient attack it compares against; if the GAN components are never actually trained, the claimed novelty and the adversarial-learning mechanism collapse.","fun_headline_variants_meta":{"raw":{"variants":["GAN attack beats FGSM and BIM on MNIST, stays natural","Invisible GAN attacks fool MNIST classifiers in 0.15s","GAN-crafted adversarial images stay natural while fooling classifiers","On MNIST, GAN attack outshines FGSM and BIM, looks real","Adversarial GAN samples: natural and fast, beat FGSM and BIM"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000303,"raw_usage":{"total_tokens":1699,"prompt_tokens":858,"completion_tokens":841,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":474,"completion_tokens_details":{"reasoning_tokens":740}},"tokens_in":474,"tokens_out":841,"duration_ms":6657,"temperature":1.0,"reasoning_tokens":740,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T10:21:38.932702+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Inspect the training loop and generated samples: if no generator or discriminator is trained, and the reported adversarial samples come from the 10-iteration clipped signed-gradient update of Eq. 3, the GAN-based claim is falsified. A direct check is to run Algorithm 1 with all GAN components removed and see whether the attack success and the 0.15-second cost are unchanged.","supporting_citations":[{"cited_title":"Adversarial examples in the physical world","cited_arxiv_id":null,"evidence_quote":"It defines the BIM baseline; the paper's Eq. 3 update is the same clipped iterative sign-gradient step."},{"cited_title":"Gener ative adversarial networks","cited_arxiv_id":null,"evidence_quote":"It supplies the generator-discriminator loss that the paper invokes as its GAN basis."},{"cited_title":"Adversarial out-domain examples for generative models","cited_arxiv_id":null,"evidence_quote":"It provides prior evidence that adversarial inputs can push pre-trained generators out of their domain, motivating the GAN-based attack."}],"review_version":1}