REVIEW 3 major objections 4 minor 18 references
Season: Spectrum-Aware Orthogonal Gradient Refinement for Transfer-Based Adversarial Attacks
T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Season, a training-free wrapper, improves transfer success of eight adversarial attacks by 6.6 percentage points on average and up to 16.0 points across CNN, ViT, and MLP targets.
desk verdict Season's frequency-decomposition wrapper is a credible incremental advance, but the 'consistently boosts' claim is contradicted by its own robustness table and untested defaults. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is a spectral–spatial–geometric pipeline applied to each raw surrogate gradient. (1) Dual-frequency decomposition: a Gaussian convolution $K_\sigma$ splits $g_t$ into low-frequency structure $g_t^L$ and high-frequency texture $g_t^H$; (2) low-saliency guidance: a one-time saliency map from a clean-image backward pass selects the bottom-15% least salient pixels, and the static weight $W = \lambda 1 + (1-\lambda)M$ with $\lambda=0.7$ attenuates high-frequency updates on foreground objects while letting them accumulate in backgrounds; (3) orthogonal projection: the accumulated high-frequency perturbation is projected to the null space of the structural perturbation, and the combined perturbation is clipped and rescaled to stay in the $\ell_\infty$ ball. The paper treats the low-frequency branch as the cross-architecture "anchor" and uses the orthogonal constraint to prevent textural noise from corrupting it.
What would settle it
Re-run the main ImageNet evaluation after replacing the low-saliency mask with a random mask of the same 15% density, keeping all other Season components fixed; if the average transfer-success gain over the eight baselines does not drop by several points, then the saliency guidance is not what carries the claimed boost, and the 'background texture' explanation is wrong.
Extended reading notes
Core claim
The central claim is that the cross-architecture transfer gap is caused by coupled gradients: standard attacks accumulate one momentum stream that mixes texture-dominated high-frequency signals with shape-dominated low-frequency signals, and the texture bias of CNN surrogates overwhelms the structural signal. Season decouples the gradient $g_t$ into a low-frequency branch $g_t^L = K_\sigma * g_t$ and a high-frequency residual $g_t^H = g_t - g_t^L$, re-weights the high-frequency branch by a static mask $W = \lambda 1 + (1-\lambda)M$ that keeps only the bottom-15% low-saliency pixels, and projects the accumulated high-frequency perturbation onto the orthogonal complement of the structural perturbation via $\hat{\delta}_t^H = \delta_t^H - \frac{\langle\delta_t^H,\delta_t^L\rangle}{\|\delta_t^L\|^2+\xi}\delta_t^L$. With these two branches maintained as separate momenta, Season replaces only the update rule of a base attack, leaving its transformation schedule and hyperparameters untouched. The paper reports that this boosts average transfer success rate by 6.6 percentage points (up to 16.0) across eight attacks and eight unseen CNN, ViT, and MLP targets, with the largest single gain on TI-FGSM (+15.5 points), and that the gains persist across five CNN surrogates and two $\ell_\infty$ budgets, albeit with small negative deltas in two of the sixteen robustness configurations.
Load-bearing premise
The load-bearing assumption is that the hand-set default hyperparameters (saliency percentile 0.15, suppression 0.7, Gaussian scale 1.0, kernel size 5) transfer across attacks, targets, and perturbation budgets, since no sensitivity analysis is reported and two configurations in the robustness study show small negative gains.
Editorial extensions
If this is right
- If Season's boost is real, any existing gradient-stabilization or input-enhancement attack can be upgraded without retraining and without changing its transformations, so the transfer attack state of the art moves up uniformly.
- The largest gain on TI-FGSM suggests that smoothing-based attacks are not losing texture information because smoothing is harmful, but because the smoothed gradient still mixes frequencies; this predicts that other smoothing methods can recover the same high-frequency capability when paired with decoupling.
- Since the mask is computed once on the clean image, the added cost is one backward pass plus a Gaussian filter and a projection per iteration, so the wrapper is practical at ImageNet scale.
- The robustness table implies the gain depends on surrogate choice and budget: two of sixteen configurations lose about 0.3–0.6 points, so the universal-boost claim is bounded rather than absolute.
Reading between the lines
- A testable extension: the same spectral split and orthogonal constraint could be applied to targeted attacks, where the structural anchor may need to be the target-class gradient rather than the source-class one.
- If the spectral-bias story is the true cause, then the optimal saliency fraction $\rho$ and suppression $\lambda$ should vary with the surrogate's measured high/low frequency ratio (1.75 for ResNet-50 vs 0.88 for ViT-B/16); Season's fixed defaults predict that a ViT surrogate needs less masking and weaker suppression, which the paper does not test.
- Because the projection is applied in perturbation space on accumulated updates rather than per-step gradients, the method implicitly assumes that $L^2$-orthogonality of cumulative updates preserves the structure; a cheaper version projecting per-step momenta might behave differently.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Season, a training-free plug-and-play wrapper for transfer-based L-infinity adversarial attacks. Season takes the surrogate gradient produced by a base attack, splits it by a Gaussian low-pass filter into low- and high-frequency branches, multiplies the high-frequency branch by a precomputed low-saliency mask (bottom 15% saliency percentile), maintains separate momentum buffers and accumulated perturbations for the two branches, projects the high-frequency perturbation onto the orthogonal complement of the low-frequency perturbation, and then clips and rescales the combined perturbation to respect the epsilon-ball. Experiments are reported on ImageNet with a ResNet-50 surrogate, eight base attacks, and eight CNN/ViT/MLP target models under a unified TPAMI protocol; the main table reports an average TSR gain of 6.6 percentage points. A robustness study across five CNN surrogates and two L-infinity budgets (Table II) and a component-wise ablation on 1,024 images (Table III) are also provided.
Significance. If the reported average gains are stable, Season would be a useful contribution: it is simple, adds small overhead, requires no training, and is compatible with a range of existing gradient-stabilization and input-enhancement attacks, with clear gains on ViT and MLP targets. The paper's strengths are the unified evaluation protocol, the breadth of target architectures, the clean ablation, and the fact that the gains are measured against external baselines. However, the universal-boost claim is currently under-supported: the paper provides no code, no error bars, no sensitivity analysis of the hand-set hyperparameters, and Table II itself contains negative deltas. These issues are addressable in a revision.
major comments (3)
- [Section IV-A and Table II] The paper's central claim that Season 'consistently boosts' eight attacks is not supported by the paper's own robustness study. With the default configuration (rho=0.15, lambda=0.7, sigma=1.0, kernel size 5), Table II reports negative deltas for DI-FGSM at 16/255 when the surrogate is DN-121 (78.4 to 78.1, -0.3pp) and GoogLeNet (71.1 to 70.5, -0.6pp). Because these are the paper's own results and no sensitivity analysis over rho, lambda, sigma, or kernel size is provided, it remains possible that the average +6.6pp gain in Table I is tied to the hand-set configuration rather than to the method. Please add a sensitivity analysis over reasonable ranges of these hyperparameters and clarify whether the default values were selected before or after inspection of the evaluation set.
- [Section III-C, Eqs. (9)-(12)] The geometric core of the method is the orthogonality constraint, but after taking signed updates (sign(m_L), sign(m_H)), clipping, and shared rescaling, the projected high-frequency perturbation no longer satisfies ⟨δ_H, δ_L⟩=0. The paper acknowledges that orthogonality is only 'approximately preserved,' but it never quantifies the approximation. Please report the empirical distribution of the cosine similarity between δ_L and δ_H (before and after the clipping/rebudgeting step) on a sample of images, or provide an argument that the residual coupling is negligible. Without this, the claimed mechanism is not verifiable from the paper's data.
- [Sections IV-A and IV-B] No error bars, confidence intervals, or repeated runs are reported for any TSR number. The differences in Table II are as small as 0.2-0.3pp on a fixed 1,024-image subset, which is within the range of sampling noise; the claim that Season is a 'consistent' booster should be supported by variance estimates or a matched-pair test, or the claim should be weakened to an average improvement.
minor comments (4)
- [Table I] In the Season+VT row, the entries '55.84', '67.27', and '61.81' are concatenated as '55.8467.2761.81'; please fix the formatting.
- [Section IV-A] Table I uses 5,000 images, while Table II and Table III use a fixed 1,024-image subset; the relationship between these subsets should be explicitly stated, and the difference in subset sizes should be acknowledged when comparing numbers across tables.
- [Eq. (5)] With the stated default lambda=0.7, high-saliency pixels receive weight 0.7 and low-saliency pixels receive weight 1.0, so the mask attenuates high-saliency updates by only 30%; the text's phrase 'redirects high-frequency perturbations to background regions' overstates the effect. Please clarify whether a stronger attenuation was considered.
- [Section III-B] The precomputed low-saliency mask is computed on the clean image and kept fixed, but for base attacks that use random input transformations (DIM, Admix, VT), the gradient is computed with respect to a transformed input; the paper should clarify how the fixed spatial mask is aligned with the transformed gradient, since the stated motivation of avoiding misalignment is not self-evident.
Circularity Check
No significant circularity: transfer gains are external, and the sole self-citation is peripheral.
full rationale
Season's central claim is an empirical transfer-success improvement on eight external ImageNet targets (RN-101, DN-121, VGG-19, MB-V2, GoogLeNet, ViT-B/16, Swin-T, Mixer-B) over eight established baselines. Equations (3)-(12) define a deterministic refinement of the surrogate gradient; none of these equations defines the reported +6.6pp average gain in terms of the method's own parameters, and no fitted parameter is renamed as a prediction. The only author-overlapping citation is [18] (FastJSMA), which appears in Related Work solely as an example of gradient decoupling for saliency-map attacks; it is not used to justify Season's mechanism or to exclude alternatives. The robustness table (Table II) shows negative deltas for DI-FGSM with DN-121 and GoogLeNet surrogates, which weakens the 'consistently boosts' wording, but underdetermination of default hyperparameters is a correctness/robustness concern, not a circularity: the paper does not fit rho, lambda, sigma on the evaluation set within the text, and no equation equates the transfer gain with an input. Therefore no circular step can be exhibited and the score is 0.
Assumptions & free parameters
free parameters (3)
- saliency percentile rho =
0.15
- suppression factor lambda =
0.7
- Gaussian scale sigma and kernel size =
sigma=1.0, k=5
assumptions (4)
- domain assumption CNN loss gradients are dominated by high-frequency textures and ViT gradients by low-frequency structure on ImageNet.
- ad hoc to paper A single fixed Gaussian scale sigma=1.0 separates structural from textural information across all images and attacks.
- domain assumption Saliency mask computed once on the clean image remains a valid guide for all perturbed iterations.
- ad hoc to paper The orthogonal projection in continuous perturbation space is approximately preserved after sign, clipping, and shared rescaling.
Cite this review
Pith. "Pith review of Season: Spectrum-Aware Orthogonal Gradient Refinement for Transfer-Based Adversarial Attacks." pith.science (2026). https://pith.science/paper/KDOHEI3Y
@misc{pith2026260804441,
author = {Pith},
title = {Pith review of: Season: Spectrum-Aware Orthogonal Gradient Refinement for Transfer-Based Adversarial Attacks},
year = {2026},
howpublished = {\url{https://pith.science/paper/KDOHEI3Y}},
note = {Machine review of arXiv:2608.04441}
}
read the original abstract
Transfer-based adversarial attacks often transfer poorly across heterogeneous architectures because CNNs favor local textures while Vision Transformers (ViTs) rely on global shapes. We propose Season, a spectrum-aware orthogonal gradient refinement framework for L-infinity transfer attacks against black-box target models on ImageNet, using a white-box surrogate. Season decomposes each update into a low-frequency branch capturing structural cues and a high-frequency branch capturing textures. A low-saliency guidance scheme reallocates high-frequency energy to background regions, preserving foreground structures that ViTs depend on. An orthogonal projection then forces the textural update to lie in the orthogonal complement of the structural direction, mitigating feature interference. As a training-free plug-and-play wrapper, Season enhances eight gradient-stabilization and input-enhancement attacks without modifying their cores. Across eight CNN, ViT, and MLP targets, Season improves transfer success rate by 6.6 percentage points on average and up to 16.0 points over strong baselines under a unified protocol.
Figures
Reference graph
Works this paper leans on
-
[1]
Explaining and harnessing adversarial examples,
Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy, “Explaining and harnessing adversarial examples,” inInternational Conference on Learning Representations (ICLR), 2015
work page 2015
-
[2]
Practical black-box attacks against machine learning,
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. Berkay Celik, and Ananthram Swami, “Practical black-box attacks against machine learning,” inProceedings of the 2017 ACM on Asia Conference on Computer and Communications Security (ASIA CCS), 2017, pp. 506–519
work page 2017
-
[3]
Boosting adversarial attacks with momentum,
Yinpeng Dong, Fangzhou Liao, Tianyu Pang, Hang Su, Jun Zhu, Xiaolin Hu, and Jianguo Li, “Boosting adversarial attacks with momentum,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 9185–9193
work page 2018
-
[4]
Improving transferability of adversarial examples with input diversity,
Cihang Xie, Zhishuai Zhang, Yuyin Zhou, Song Bai, Jianyu Wang, Zhou Ren, and Alan L. Yuille, “Improving transferability of adversarial examples with input diversity,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 2730–2739
work page 2019
-
[5]
Re- visiting transferable adversarial images: Systemization, evaluation, and new insights,
Zhengyu Zhao, Hanwei Zhang, Renjue Li, Ronan Sicre, Laurent Am- saleg, Michael Backes, Qi Li, Qian Wang, and Chao Shen, “Re- visiting transferable adversarial images: Systemization, evaluation, and new insights,”IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 48, no. 1, pp. 765–780, Jan 2026
work page 2026
-
[6]
Robert Geirhos, Patricia Rubisch, Claudio Michaelis, Matthias Bethge, Felix A. Wichmann, and Wieland Brendel, “ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness,” inInternational Conference on Learning Representations (ICLR), 2019
work page 2019
-
[7]
An image is worth 16x16 words: Transformers for image recognition at scale,
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weis- senborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby, “An image is worth 16x16 words: Transformers for image recognition at scale,” inInternational Conference on Learning Representations (ICLR), 2021
2021
-
[8]
Intriguing properties of vision transformers,
Muzammal Naseer, Kanchana Ranasinghe, Salman Khan, Munawar Hayat, Fahad Shahbaz Khan, and Ming-Hsuan Yang, “Intriguing properties of vision transformers,” inAdvances in Neural Information Processing Systems (NeurIPS), 2021, vol. 34, pp. 23296–23308
work page 2021
Show all 18 references
-
[9]
Nesterov accelerated gradient and scale invariance for adversarial attacks,
Jianyu Lin, Chuan Gan, and Song Han, “Nesterov accelerated gradient and scale invariance for adversarial attacks,” inInternational Conference on Learning Representations (ICLR), 2020
2020
-
[10]
Patch-wise attack for fooling deep neural network,
Lianli Gao, Qilong Zhang, Jingkuan Song, Xianglong Liu, and Heng Tao Shen, “Patch-wise attack for fooling deep neural network,” inComputer Vision – ECCV 2020. 2020, vol. 12373 ofLecture Notes in Computer Science, pp. 307–322, Springer
2020
-
[11]
Enhancing the transferability of adversarial attacks through variance tuning,
Xiaosen Wang and Kun He, “Enhancing the transferability of adversarial attacks through variance tuning,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 1924–1933
2021
-
[12]
Evading defenses to transferable adversarial examples by translation-invariant attacks,
Yinpeng Dong, Tianyu Pang, Hang Su, and Jun Zhu, “Evading defenses to transferable adversarial examples by translation-invariant attacks,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2019, pp. 4312–4321
2019
-
[13]
Admix: Enhancing the transferability of adversarial attacks,
Xiaosen Wang, Xuanran He, Jingdong Wang, and Kun He, “Admix: Enhancing the transferability of adversarial attacks,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 16138–16147
2021
-
[14]
Skip connections matter: On the transferability of adversarial examples generated with ResNets,
Dongxian Wu, Yisen Wang, Shu-Tao Xia, James Bailey, and Xingjun Ma, “Skip connections matter: On the transferability of adversarial examples generated with ResNets,” inInternational Conference on Learning Representations (ICLR), 2020
2020
-
[15]
Backpropagating linearly improves transferability of adversarial examples,
Yiwen Guo, Qizhang Li, and Hao Chen, “Backpropagating linearly improves transferability of adversarial examples,” inAdvances in Neural Information Processing Systems (NeurIPS), 2020
2020
-
[16]
Feature importance-aware transferable adversarial attacks,
Zhibo Wang, Hengchang Guo, Zhifei Zhang, Wenxin Liu, Zhan Qin, and Kui Ren, “Feature importance-aware transferable adversarial attacks,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2021, pp. 7619–7628
2021
-
[17]
Improving adversarial trans- ferability via neuron attribution-based attacks,
Jianping Zhang, Weibin Wu, Jen tse Huang, Yizhan Huang, Wenxuan Wang, Yuxin Su, and Michael R. Lyu, “Improving adversarial trans- ferability via neuron attribution-based attacks,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022,...
2022
-
[18]
Fastjsma: Accelerating jacobian- based saliency map attacks through gradient decoupling,
Zhenghao Gao, Shengjie Xu, Zijing Li, Meixi Chen, Chaojian Yu, Yuanjie Shao, and Changxin Gao, “Fastjsma: Accelerating jacobian- based saliency map attacks through gradient decoupling,” inProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2...
2025
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.