{"id":"36a371d6-703d-4cb7-9d79-e57ffc73451d","arxiv_id":"2502.03758","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A prompting defense that learns per-class phase and amplitude Fourier prompts and weights them by robust accuracy improves adversarial robustness of frozen classifiers.","lead":"This paper proposes a defense that adds learned per-class prompts in the Fourier phase and amplitude spectra to a frozen image classifier, boosting its accuracy on adversarial images. The method is tested on CIFAR-10 and Tiny-ImageNet and reports large robustness gains over existing prompt-based defenses, though with a drop in clean accuracy on naturally trained models.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Adaptive attack Eq. 10 optimizes with the ground-truth prompt, not the predicted-label prompt selection used at test time, so the headline robustness numbers may not survive a fully adaptive attacker.","rationale":"Agree with the reader's weakest assumption. The concern is load-bearing because the claimed large margin is precisely against white-box attacks, and the only white-box adaptive evaluation omits the unique test-time mechanism, namely predicted-label prompt selection. If the proposed test shows a large drop, the headline numbers do not support the conclusion and the contribution reduces to a non-adaptive defense. Independent positives exist: extensive tables across two datasets and several model types, transferability experiments, ablations, and an explicit attempt at adaptive attacks; however, there is no code release and no machine-checked verification. A secondary issue is the missing phase-replacement baseline: Table 1 suggests replacing the adversarial phase with a natural phase gives high robustness, so the mechanism attribution needs this baseline to show learned per-class prompts add value beyond fixed frequency replacement. Still, the adaptive-attack gap is the first thing to settle, and the verdict should remain conditional until that check is run.","tokens_in":16948,"tokens_out":6602,"duration_ms":76045,"concrete_test":"Run a fully adaptive white-box attack on the actual test-time defense. Replace the ground-truth-prompt objective in Eq. 10 with the test-time selection rule: for a candidate delta, set y_pred = argmax_c h_theta(x+delta) (or use the current forward prediction), then compute the loss CE(h_theta(F^{-1}(F_phi(x+delta)+p_phi^{y_pred}, F_xi(x+delta)+w^* p_xi^{y_pred})), y), and optimize delta by PGD/APGD (20 and 40 iterations) with straight-through or softmax-blend gradients through the argmax. If NAT/AT CIFAR-10 robust accuracy drops materially below the reported 37.34%/52.31% (e.g., below 10% for NAT), then the claimed large robustness margin is an artifact of the non-adaptive evaluation. Also run the same attack with the worst-case prompt choice max_{y'} CE(... p^{y'} ...) to upper-bound the threat.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's central numerical claims (e.g., NAT+PAP 37.34% AA on CIFAR-10 vs 0.61% for C-AVP) rest on white-box evaluations. At test time the defense is two-stage: y_pred = h_theta(x) is used to select prompts p_phi^{y_pred}, p_xi^{y_pred}, and the prompted image is reclassified. The adaptive attack in Eq. 10, however, optimizes delta against CE(F^{-1}(F_phi(x+delta)+p_phi^y, F_xi(x+delta)+w_t p_xi^y), y), i.e., the ground-truth class prompt, and Section 4.4 evaluates against this fixed-prompt objective. This is not the deployed classifier. A fully informed attacker can instead try to drive y_pred to a class whose prompt is harmful, then exploit the resulting misclassification; the data-prompt mismatching loss (Eq. 5) is trained on randomly sampled wrong classes and provides no worst-case guarantee over the adversarially selected class. Thus Tables 3 and 7 may overstate robustness. The weighting scheme (Eq. 6) and learned w* are also knowable by the attacker, so gradient-based attacks on the full two-stage function with a soft relaxation of the argmax are feasible. This is the load-bearing gap between the paper's claim and its evidence.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a prompt-based adversarial defense called Phase and Amplitude-aware Prompting (PAP). For each class, the method learns a phase-level prompt and an amplitude-level prompt in the Fourier domain; at test time, prompts are selected by the model's predicted label and added to the input's phase and amplitude spectra before reclassification. The amplitude-prompt weight is updated during training according to the ratio of robust accuracies obtained under amplitude-only and phase-only prompting. A data-prompt mismatching loss is introduced to tolerate the mismatch between a test image and the prompt selected by its predicted label. Experiments on CIFAR-10 and Tiny-ImageNet with NAT, AT, TRADES, and MART pre-trained models report large gains over C-AVP and Freq baseline defenses under AutoAttack, C&W, and DDN, plus transferability results and an adaptive-attack evaluation.","tokens_in":17277,"tokens_out":4391,"duration_ms":47088,"significance":"If the results hold, the paper offers a novel and efficient prompt-based defense direction by explicitly separating phase and amplitude semantic patterns, and it provides unusually large robustness improvements (e.g., 37.34% vs 0.61% AutoAttack accuracy on naturally pre-trained ResNet18 CIFAR-10). The experimental scope is broad: three attack types, two datasets, four training regimes, and cross-model transfer including a vision transformer. The paper is also honest about limitations such as natural-accuracy loss and lack of ImageNet-scale evaluation. However, the central robustness claim rests on an adaptive-attack evaluation that does not target the actual deployed classifier, and for naturally pre-trained models the learned amplitude weight collapses to zero, so the 'amplitude-aware' component is not operational in a major portion of the experiments. The contribution is therefore promising but needs essential revisions.","major_comments":[{"comment":"The adaptive attack maximizes the cross-entropy loss against a prompted image formed with the ground-truth class prompts p_phi^y and p_xi^y. This does not match the deployed test-time procedure in Eq. (9), which selects prompts according to the model's predicted label y_pred. A fully informed adversary can instead attempt to drive y_pred to a class whose prompts are harmful, or differentiate through a soft relaxation of the argmax selection. As written, Eq. (10) evaluates a fixed-prompt classifier, not the proposed two-stage defense, so Table 7 does not establish robustness under a fully adaptive attacker. The claimed margins in Tables 3 and 7 may be substantially overstated.","section":"§4.4, Eq. (10)"},{"comment":"The data-prompt mismatching loss trains on randomly sampled wrong-class prompts, but at test time the selected prompt class is the model's predicted label, which an adversary can manipulate. Random negative classes provide no worst-case guarantee against an attacker who explicitly forces y_pred to a class with a harmful prompt. The paper should either train mismatching loss on adversarial predicted-label mismatches or provide a rigorous argument that random mismatches cover the adversarial case. This gap is load-bearing because the practical robustness of the defense depends on the assumption that mismatching is benign.","section":"§3.3.4 and Eq. (5)"},{"comment":"The 'white-box' results in Table 3 are apparently obtained by crafting adversarial perturbations on the original, unprompted model and then applying the prompt afterward. This is a non-adaptive attack setup: the attacker does not have access to the prompting module or the learned prompts. The paper should state this explicitly and clearly differentiate these numbers from the adaptive-attack results in Table 7. The very large margins over baselines (e.g., 80.27% vs 1.93% on C&W for NAT ResNet18) are expected under non-adaptive attacks and do not by themselves support white-box robustness in the standard sense.","section":"§4.2 and Table 3"},{"comment":"Table 5 reports w* = 0 for all naturally pre-trained models on both datasets. With w=0, Eq. (7) reduces to phase-only prompting, so the amplitude-level prompts are completely suppressed for NAT models. This contradicts the paper's title and the motivation that amplitude-aware prompting is beneficial. The paper needs to discuss this behavior explicitly, provide a phase-only baseline in the ablations, and either justify the collapse to zero or impose a floor on w. Without this, the claim that both phase and amplitude components contribute is unsupported for a large part of the evaluation.","section":"Table 5 and §3.3.2"}],"minor_comments":[{"comment":"The symbol y_pred is used in Eq. (9) but never formally defined; please state that it is the output of the target model h_theta on the clean test image (or on the original input before prompting).","section":"§3.3.4"},{"comment":"The notation h^{y'_i}_theta and h^{y_i}_theta is used without defining that these denote the output logits for the corresponding classes; please clarify in the text.","section":"Eq. (5)"},{"comment":"The text says 'The weight strategy is specified as' but should read 'The weighting strategy'; also, the update in Eq. (6) depends on the initial weight w_0, which is not specified in the experimental settings. Please report the initialization and its sensitivity.","section":"§3.3.2"},{"comment":"The phrase 'we retrain our PAP on this attack' is ambiguous: it should clarify whether the prompts, the weight w*, or both are retrained under the adaptive objective, and whether baselines are retrained with the same number of iterations.","section":"§4.4"},{"comment":"The black-box results use transfer-based attacks from a VGG19 surrogate; the paper should state whether the surrogate was trained independently on the same dataset and whether the prompting defense is included in the surrogate's gradient computation.","section":"Appendix D, Table 12"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid empirical core and a fresh idea, but the main robustness evidence has a methodological gap: the adaptive attack does not target the deployed predicted-label prompt selection. This is fixable within the paper's scope (e.g., by adding an attack that optimizes through a soft argmax or by evaluating against a worst-case class selection), but it is essential before publication. The zero learned weight for NAT models also raises a conceptual issue that the authors should address head-on. I would be willing to review a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Xu and colleagues present PAP, a test-time prompt-based defense that learns per-class Fourier phase and amplitude prompts and selects prompts at test time via the model's predicted label. The headline results are strong: on CIFAR-10 with a naturally trained ResNet18, PAP lifts AutoAttack accuracy from 0.61% (C-AVP) to 37.34%, and C&W from 1.93% to 80.27%. Those gains hold across AT, TRADES, and MART models, and the prompts transfer to other architectures including a vision transformer. The ablations are thorough: each loss term matters, the amplitude weight helps, per-class prompts beat a universal prompt, and blurring edges does not erase the gains. This is a genuinely useful empirical contribution to prompt-based defenses.\n\nThe main soft spot is the adaptive attack. At test time the defense is two-stage: the model's first pass selects a class, and prompts from that class are applied before a second pass gives the final prediction. The adaptive attack in Equation 10 optimizes the perturbation against the ground-truth class's prompts, not the predicted-label selection. That is not the deployed function. A fully informed attacker could try to steer the first-stage prediction to a class whose prompt is harmful, and the data-prompt mismatching loss offers no worst-case guarantee against an adversarially chosen class. So the numbers in Table 7 may overstate robustness. The authors should redo the adaptive evaluation with a soft approximation of the selection operation, or at least discuss this gap. This is not a reason to reject, but it is a reason to be skeptical of the absolute robustness claims until fixed.\n\nA second, smaller issue: the motivation section shows that replacing phase or amplitude spectra with natural ones improves robustness, yet the experiments never compare PAP against that simple amplitude-phase recombination baseline (from the cited [24]). Adding that baseline would clarify whether the learned prompts add anything beyond spectrum replacement. No code is released, which makes replication harder, but the method is simple enough to reimplement. The limitations section honestly notes the natural accuracy drop and the lack of ImageNet evaluation; neither is damning.\n\nOverall, the paper is coherent and the empirical work is above the usual bar. The central heuristic is plausible and the ablations support it. The adaptive attack gap is the one load-bearing issue. This paper deserves a serious referee and, with a stronger adaptive attack and a code release, could be a solid venue-level contribution. I would send it to review with a request for major revision on the evaluation.","headline":"Strong empirical prompt-based defense whose adaptive-attack evaluation does not match the deployed prompt-selection procedure; deserves review with a request for a fully adaptive attack and code release.","tokens_in":17782,"tokens_out":4931,"would_cite":false,"duration_ms":48975,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that adding per-class Fourier phase and amplitude prompts to images before classification stabilizes model predictions and improves adversarial robustness by large margins over existing prompt-based defenses, without…","keywords":["adversarial robustness","prompt-based defense","phase spectrum","amplitude spectrum","Fourier transform","visual prompting","test-time defense","transferability"],"falsifier":"Run an adaptive attack that at each optimization step uses the model's actual predicted label to choose the phase and amplitude prompts (Equation 9 instead of Equation 10) and check whether the reported AutoAttack accuracy on CIFAR-10 NAT+PAP stays above 30%; if it collapses to the C-AVP level, the predicted-label selection is the weak link.","tokens_in":16731,"feed_emoji":"🛡️","tokens_out":10852,"duration_ms":93540,"temperature":0.7,"pith_summary":"The paper is trying to establish that adversarial robustness can be improved by prompting images with semantic-specific Fourier patterns rather than mixed pixel or frequency patterns. It claims that learning one phase-level and one amplitude-level prompt per class, and weighting the amplitude prompt by the ratio of robust accuracies each prompt type achieves during training, lets a fixed model resist AutoAttack, C&W, and DDN attacks substantially better than the C-AVP and Freq prompting baselines. If true, this offers a cheap complement to adversarial training: the target network's weights stay frozen and no purification module is needed. On CIFAR-10 with a naturally trained ResNet18, AutoAttack accuracy rises from 0.61% with C-AVP to 37.34% with PAP, and the prompts transfer to other architectures. The paper acknowledges that this gain costs some natural accuracy on naturally pre-trained models, and it reports no ImageNet evaluation.","feed_headline":"Fourier prompts push CIFAR-10 AutoAttack robustness to 37%","feed_subtitle":"A test-time prompting defense that needs no model retraining beats prior methods on CIFAR-10 and Tiny-ImageNet.","key_machinery":"The machinery is the phase/amplitude Fourier prompting scheme together with the robustness-based weighting rule. For each class $y$, a phase prompt $p_{\\phi, y}$ and an amplitude prompt $p_{\\xi, y}$ are initialized from the spectra of a random natural example, and a prompted image is formed by $x^p = \\mathcal{F}^{-1}(\\phi_x + p_{\\phi, y_{pred}}, \\xi_x + w^* p_{\\xi, y_{pred}})$. The amplitude weight is updated every few epochs as $w_t = w_{t-1} \\times A_{\\xi}/A_{\\phi}$, where $A_{\\xi}$ and $A_{\\phi}$ are the robust accuracies on adversarial training examples obtained with amplitude-only and phase-only prompting. This ratio is the load-bearing mechanism: it converts the relative usefulness of the two spectral prompt types into a scalar that controls how strongly amplitude information is added. A data-prompt mismatching loss ensures that even when the predicted label is wrong, the selected prompt does not cause a confident misclassification.","core_discovery":"The central claim is that disentangling prompts into the phase and amplitude spectra of the Fourier transform is the key to prompt-based defenses. Phase spectra carry structure and amplitude spectra carry texture, and the paper argues that previous prompting on mixed pixel or frequency patterns fails to focus on these semantics. PAP learns per-class prompts $p_{\\phi}$ and $p_{\\xi}$, forms the prompted image $x^p = \\mathcal{F}^{-1}(\\phi_{\\tilde{x}} + p_{\\phi, y_{pred}}, \\xi_{\\tilde{x}} + w^* p_{\\xi, y_{pred}})$, and during training adjusts the amplitude weight $w_t$ by the ratio of robust accuracies obtained when only the amplitude prompt or only the phase prompt is applied. At test time the prompt is selected by the model's own predicted label, and a data-prompt mismatching loss is trained so that prompts from wrong classes do not wreck predictions. The discovery is that this scheme yields large robustness gains on both naturally and adversarially pre-trained models, with the learned weight automatically diminishing amplitude prompting when it is not helpful.","pith_inferences":["A finer-grained weighting, per class or per image instead of one global scalar updated every five epochs, could recover some of the natural accuracy that the paper reports losing on naturally pre-trained models; the paper does not test this.","The adaptive attack in Section 4.4 optimizes against prompts taken from the ground-truth label, whereas test-time prompts are chosen from the predicted label; an adaptive adversary that targets the actual selection rule would be a sharper test that the paper does not report.","Because phase spectra carry structure, the same prompting-plus-weighting design could be plugged into other structure-preserving decompositions such as wavelets, which the paper does not explore."],"forward_implications":["Because the target model's weights stay frozen, PAP can be layered on both naturally and adversarially pre-trained models, improving AutoAttack robustness over the base model in every reported setting.","Prompts trained on ResNet18 transfer to VGG19, WRN28-10, and Swin Transformer, so the defense can be trained once and applied across architectures.","The predicted-label selection rule needs only one prompt pair per test image instead of traversing all classes, making the method practical on datasets with many classes such as Tiny-ImageNet.","The learned amplitude weight drops to zero on naturally pre-trained CIFAR-10 models, showing the defense can automatically discover that phase-only prompting is the useful component in that regime."],"supporting_citations":[{"why":"provides the C-AVP pixel-domain prompting baseline that PAP is compared against.","marker":"[12]"},{"why":"provides the Freq frequency-domain prompting baseline and its masked prompting strategy.","marker":"[11]"},{"why":"supplies the PGD attack used to craft adversarial examples during prompt training.","marker":"[7]"},{"why":"establishes that phase spectra relate to structure and amplitude spectra to texture, motivating the prompt design.","marker":"[24]"},{"why":"defines the AutoAttack used for L-infinity robustness evaluation.","marker":"[25]"},{"why":"defines the C&W L2 attack used for robustness evaluation.","marker":"[26]"},{"why":"defines the DDN L2 attack used for robustness evaluation.","marker":"[27]"},{"why":"supplies the TRADES adversarially pre-trained model on which PAP is evaluated.","marker":"[29]"},{"why":"supplies the MART adversarially pre-trained model on which PAP is evaluated.","marker":"[30]"}],"fun_headline_variants":["Fourier phase and amplitude prompts: robust without retraining","Phase and amplitude prompts: adversarial defense without retraining","Fourier prompt defense: phase+amplitude improves robust accuracy","Phase-aware prompts: no retraining, robust to AutoAttack","Phase-amplitude prompts hit 37% AutoAttack on CIFAR-10"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes a wrong predicted label at test time will still select a helpful prompt rather than a harmful one, because the data-prompt mismatching loss only trains against randomly sampled wrong labels.","fun_headline_variants_meta":{"raw":{"variants":["Fourier phase and amplitude prompts: robust without retraining","Phase and amplitude prompts: adversarial defense without retraining","Fourier prompt defense: phase+amplitude improves robust accuracy","Phase-aware prompts: no retraining, robust to AutoAttack","Phase-amplitude prompts hit 37% AutoAttack on CIFAR-10"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00075,"raw_usage":{"total_tokens":3323,"prompt_tokens":911,"completion_tokens":2412,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":527,"completion_tokens_details":{"reasoning_tokens":2322}},"tokens_in":527,"tokens_out":2412,"duration_ms":18385,"temperature":1.0,"reasoning_tokens":2322,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T00:50:35.244261+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run an adaptive attack that at each optimization step uses the model's actual predicted label to choose the phase and amplitude prompts (Equation 9 instead of Equation 10) and check whether the reported AutoAttack accuracy on CIFAR-10 NAT+PAP stays above 30%; if it collapses to the C-AVP level, the predicted-label selection is the weak link.","supporting_citations":[{"cited_title":"Visual prompting for adversarial robustness","cited_arxiv_id":null,"evidence_quote":"provides the C-AVP pixel-domain prompting baseline that PAP is compared against."},{"cited_title":"Improving adversarial robustness of masked autoencoders via test-time frequency-domain prompting","cited_arxiv_id":null,"evidence_quote":"provides the Freq frequency-domain prompting baseline and its masked prompting strategy."},{"cited_title":"Towards deep learning models resistant to adversarial attacks","cited_arxiv_id":null,"evidence_quote":"supplies the PGD attack used to craft adversarial examples during prompt training."},{"cited_title":"Amplitude-phase recombination: Rethinking robustness of convolutional neural networks in frequency domain","cited_arxiv_id":null,"evidence_quote":"establishes that phase spectra relate to structure and amplitude spectra to texture, motivating the prompt design."},{"cited_title":"Decoupling direction and norm for efficient gradient-based l2 adversarial attacks and defenses","cited_arxiv_id":null,"evidence_quote":"defines the DDN L2 attack used for robustness evaluation."},{"cited_title":"Improving adversarial robustness requires revisiting misclassified examples","cited_arxiv_id":null,"evidence_quote":"supplies the MART adversarially pre-trained model on which PAP is evaluated."}],"review_version":1}