REVIEW 4 major objections 7 minor 42 references
Theoretical Corrections and the Leveraging of Reinforcement Learning to Enhance Triangle Attack
T0 review · 4 major / 7 minor · reviewed 2026-08-12 · deepseek-v4-flash
Pith's one-line read This paper claims that Triangle Attack's fixed rule for updating its learned angle can be replaced by an online Q-learning policy, giving an attack that matches the original's success rate with half as many queries.
desk verdict A modest, honest incremental attack paper whose headline query-efficiency claim is not actually demonstrated: TARL@500 is never compared against TA@500, so the RL contribution is unmeasured. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying object is the learned angle $\alpha$ in Triangle Attack's triangle construction, where the benign image, the current adversarial candidate, and the next candidate form a triangle in a 2D subspace of the image's frequency space. $\alpha$ controls the trade-off between finding any adversarial candidate and minimizing the perturbation, and TARL's machinery is a Q-table with discretized $\alpha$ states and two actions (increase or decrease $\alpha$), updated by epsilon-greedy Q-learning with reward $-\ell_2$ for successful adversarial candidates and 0 for failures. This learned update replaces TA's fixed multiplicative $\alpha$ rule, and it is the component the paper credits for making the attack adaptive to different decision-boundary shapes and more query-efficient.
What would settle it
Run TARL on the same 1,000 ImageNet images and models with the Q-table update disabled, so each alpha action is chosen randomly from the same epsilon-greedy distribution; if attack success rates at 500 queries stay within noise of TARL's reported numbers, then the learned policy is not what produces the query savings.
Extended reading notes
Core claim
In the paper's own framing, the discovery is that Triangle Attack's $\alpha$-update proposition does not hold for every decision-boundary shape, and that this failure can be repaired by learning. TARL replaces TA's deterministic rule (increase $\alpha$ on adversarial queries, decrease it on failures, at fixed rates) with a Q-learning agent whose state is the current $\alpha$ value plus the observed $\ell_2$ distance and success label, whose actions are increase or decrease $\alpha$, and whose reward is the negative $\ell_2$ norm for adversarial candidates and zero otherwise. The agent is updated online after every query using the standard Q-learning rule, with epsilon-greedy exploration and clipping of $\alpha$ to a predefined range. The reported conclusion is that, across ImageNet and CIFAR-10 classifiers including vision transformers, a state-space model, and a diffusion-based defense, TARL at a 500-query budget performs within a few percent of TA at a 1,000-query budget, and outperforms TA on every ImageNet model when the perturbation limit is set to RMSE < 0.5.
Load-bearing premise
The attack's gains rest on the assumption that a Q-learning agent, learning from scratch for each image, can converge to a useful alpha-adjustment policy within the 500-query budget even though the decision boundary and the best alpha keep changing as the attack progresses.
Editorial extensions
If this is right
- If TARL's learning works as claimed, a 500-query budget is enough to reach attack success rates that previously required 1,000 queries, effectively doubling the number of attacks an adversary can attempt under query-limited conditions.
- The gain is budget-dependent: at RMSE C=0.01, TARL trails TA by 1.9 to 7.3 percentage points on ImageNet models, so tight-perturbation settings still favor TA's fixed rule.
- TARL stays within about 2 percentage points of TA against a diffusion-based adversarial training defense while using half the queries, extending query-efficient attacks to at least one state-of-the-art defense.
- Because TARL only changes the alpha-update step, its Q-learning policy can be layered onto any future improvement to TA's subspace selection or beta search without changing the rest of the attack.
Reading between the lines
- A direct test the paper leaves implicit is to train the Q-agent offline on a set of images and freeze it before attacking new images; the current per-image online learning makes it hard to separate genuine learned boundary geometry from the effect of random exploration within the first 500 queries.
- The crossover pattern in the tables suggests an adaptive attack that switches between TA's fixed rule and TARL's learned rule depending on the perturbation budget could dominate either method alone, since each dominates in a different RMSE regime.
- The Q-table discretizes alpha even though the paper describes alpha as continuous; if a finer alpha grid or a continuous function approximator closes the remaining small gaps at RMSE C=0.05, that would indicate discretization, not learning, is the current bottleneck.
- A cross-image transfer test would clarify whether the agent learns reusable boundary geometry or only per-image quirks; TARL's stated per-image training leaves this question open.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes TARL, a decision-based black-box attack that replaces Triangle Attack's fixed alpha-update rule with an online Q-learning policy, and reports experiments comparing TARL at 500 queries to TA at 1,000 queries across ImageNet and CIFAR-10 models, including a diffusion-based defense. The paper also presents illustrative counterexamples to Triangle Attack's Proposition 1 about the learned angle alpha and claims that TARL attains similar or better attack success rates with half the queries. The empirical tables show that TARL is consistently below TA on tight perturbation budgets and only surpasses TA at the largest tested budget (RMSE C = 0.5).
Significance. If the central claim were established, TARL would be a useful query-efficiency improvement over a competitive decision-based attack, and the extension to nine ImageNet/CIFAR-10 models plus a diffusion defense is a broader evaluation than the original Triangle Attack paper. I see no circularity problem in comparing TARL to external attacks, since the agent learns online from the attack's own queries rather than from benchmark labels. However, the claimed advantage is not currently supported because the experimental design lacks a matched-query baseline and the algorithm description contains unresolved inconsistencies; the significance is therefore conditional on substantial revision.
major comments (4)
- [§4.2, Tables 2–4] The abstract's 'similar, if not better' claim is not supported for tight budgets: across ImageNet models, TARL at 500 queries is 1.9–7.3 percentage points below TA at 1,000 queries for C=0.01, 1.2–4.7 points for C=0.05, and up to 4.5 points for C=0.1, with TARL exceeding TA only at C=0.5 in Table 5. Because the query budgets differ in every comparison, the contribution of the RL component is confounded with the query-budget reduction; please report TA at 500 queries under matched conditions, since without that control the claimed 'half as many queries' advantage is not measurable.
- [§2.3 and Algorithms 1–4] The algorithm description is internally inconsistent: Algorithm 1 line 7 checks 'if q-table is empty' although Algorithm 2 initializes a zero-filled Q-table; Algorithm 4 assigns reward = -l2 for adversarial outcomes while Section 2.3 states reward = 1/l2; Algorithm 3 uses an undefined probability p and an action list that is never constructed; and the text claims Q-learning handles continuous action spaces while Algorithm 2 builds a discrete state-action table. As written, the reported numbers cannot be attributed to the described learning procedure, so please provide consistent pseudocode, define every symbol, and state the discretization of the action space.
- [§2.2] The claimed 'theoretical corrections' are illustrated with 2D figures rather than formal arguments; no theorem or counterexample is stated, and the conclusion that the alpha-update algorithm 'shows there is potential for improvement' is not derived from the figures alone. If the paper's title promises corrections, please state the assumptions and provide a formal counterexample to Proposition 1 of [37], or alternatively reframe the section as an empirical limitation.
- [§4.1 and Tables 1–5] No hyperparameter values, exploration schedules, random seeds, or variance estimates are reported, yet several ASR differences in Tables 4 and 5 are 0.1–0.9 percentage points, which is within plausible run-to-run noise for the reported sample sizes. Please provide the Q-learning hyperparameters (learning rate, discount factor, epsilon schedule, alpha step size, alpha range), the number of independent runs, and standard deviations or confidence intervals for every table.
minor comments (7)
- [§1.1] There are typos such as 'threating' for 'threatening' and 'sucessful' for 'successful'; please proofread the manuscript.
- [§2.2 and Figures 1–2] The text refers to 'Figure 1 Part (a)' and 'Figure 2 Part (b)', but the figures are not included in the manuscript; please ensure all figures and their sub-panels are present and labeled.
- [Tables 2–5] The sign convention of the 'Diff.' column should be stated in each table caption; the text says a positive difference means TARL generated more adversarial examples, but the captions do not say this.
- [§4.2] The conclusion says the evaluation was extended to 'the entire ImageNet dataset', but the experiments use 1,000 selected images per model; please rephrase to avoid overstating the evaluation scope.
- [§4.1, Eq. (3)] Table 5 is described as 'RMSE < 0.5', while Eq. (3) defines RMSE with a constant C; please specify that Table 5 uses C = 0.5 and clarify the relationship between C and the RMSE threshold.
- [References] Reference [15] is cited for Inception-v3 but points to Ioffe's batch normalization paper; please replace it with the correct Inception-v3 citation.
- [§4.1] The contribution list mentions '10 victim models and 1 defense model', but Tables 1–5 include 11 entries including the diffusion defense; please reconcile the count.
Circularity Check
No circular derivation; TARL is an empirical RL modification of an external attack, and the main weakness is a missing matched-query control, not circularity.
full rationale
TARL is evaluated head-to-head against TA and several external decision-based attacks on the same images, models, and RMSE budgets (Section 4.2). The RL agent's reward is based on the l2 distance of the candidate perturbation, which is the quantity the attack optimizes; this is an objective choice, not a parameter fitted to the reported ASR values. The paper inherits the DCT subspace construction, beta search, and binary-search initialization from the external TA paper [37] rather than from a self-citation chain; the only self-citations ([24], [25]) are background references on black-box attacks and ViT robustness and are not load-bearing. The 'theoretical correction' in Section 2.2 is informal and illustrative, not a formal derivation, so no uniqueness theorem or ansatz is imported from the authors' own prior work. The experiments also compare TARL at 500 queries against TA at 1000 queries without reporting TA at 500 queries, which weakens the query-efficiency claim as a controlled attribution; however, that is an experimental-control issue, not circularity. No circular step was found that reduces a claimed result to its own inputs.
Assumptions & free parameters
free parameters (5)
- Q-learning learning rate =
not reported
- Discount factor =
not reported
- Exploration rate (epsilon) =
not reported
- Alpha step size =
not reported
- Alpha range (alpha_min, alpha_max) =
not reported
assumptions (3)
- domain assumption Assumption 1: there exists an adversarial example within the perturbation budget.
- domain assumption DCT-based 2D subspace preserves the critical image geometry.
- ad hoc to paper Online Q-learning converges to a useful alpha-update policy within the query budget.
Cite this review
Pith. "Pith review of Theoretical Corrections and the Leveraging of Reinforcement Learning to Enhance Triangle Attack." pith.science (2026). https://pith.science/paper/AEARVIFC
@misc{pith2026241112071,
author = {Pith},
title = {Pith review of: Theoretical Corrections and the Leveraging of Reinforcement Learning to Enhance Triangle Attack},
year = {2026},
howpublished = {\url{https://pith.science/paper/AEARVIFC}},
note = {Machine review of arXiv:2411.12071}
}
read the original abstract
Adversarial examples represent a serious issue for the application of machine learning models in many sensitive domains. For generating adversarial examples, decision based black-box attacks are one of the most practical techniques as they only require query access to the model. One of the most recently proposed state-of-the-art decision based black-box attacks is Triangle Attack (TA). In this paper, we offer a high-level description of TA and explain potential theoretical limitations. We then propose a new decision based black-box attack, Triangle Attack with Reinforcement Learning (TARL). Our new attack addresses the limits of TA by leveraging reinforcement learning. This creates an attack that can achieve similar, if not better, attack accuracy than TA with half as many queries on state-of-the-art classifiers and defenses across ImageNet and CIFAR-10.
Figures
Reference graph
Works this paper leans on
-
[37]
Triangle attack: A query-efficient decision- based adversarial attack
Xiaosen Wang, Zeliang Zhang, Kangheng Tong, Dihong Gon g, Kun He, Zhifeng Li, and Wei Liu. Triangle attack: A query-efficient decision- based adversarial attack. In European conference on computer vision , pages 156–174. Springer,
-
[1]
IEEE transactions on Computers , 100(1):90–93, 1974
Discrete cosine transform. IEEE transactions on Computers , 100(1):90–93, 1974. 4
work page 1974
-
[2]
Square attack: a query-efficient black-box adversaria l attack via random 16 search
Maksym Andriushchenko, Francesco Croce, Nicolas Flamm arion, and Matthias Hein. Square attack: a query-efficient black-box adversaria l attack via random 16 search. In European conference on computer vision , pages 484–501. Springer,
-
[3]
Anish Athalye, Nicholas Carlini, and David Wagner. Obfu scated gradients give a false sense of security: Circumventing defenses to advers arial examples. In International conference on machine learning , pages 274–283. PMLR, 2018. 8
work page 2018
-
[4]
Beit: Ber t pre-training of image transformers
Hangbo Bao, Li Dong, Songhao Piao, and Furu Wei. Beit: Ber t pre-training of image transformers. arXiv preprint arXiv:2106.08254 , 2021. 11
arXiv 2021
-
[5]
Dec ision-based adver- sarial attacks: Reliable attacks against black-box machin e learning models
Wieland Brendel, Jonas Rauber, and Matthias Bethge. Dec ision-based adver- sarial attacks: Reliable attacks against black-box machin e learning models. In International Conference on Learning Representations , 2018. 2, 3, 8
work page 2018
-
[6]
Towards evaluating t he robustness of neural networks
Nicholas Carlini and David Wagner. Towards evaluating t he robustness of neural networks. In 2017 ieee symposium on security and privacy (sp) , pages 39–57. Ieee,
work page 2017
-
[7]
H opskipjumpattack: A query-efficient decision-based attack
Jianbo Chen, Michael I Jordan, and Martin J Wainwright. H opskipjumpattack: A query-efficient decision-based attack. In 2020 ieee symposium on security and privacy (sp) , pages 1277–1294. IEEE, 2020. 3, 13
work page 2020
Show all 42 references
-
[8]
Zoo: Zeroth order optimization based black-box attacks to deep n eural networks with- out training substitute models
Pin-Yu Chen, Huan Zhang, Yash Sharma, Jinfeng Yi, and Cho -Jui Hsieh. Zoo: Zeroth order optimization based black-box attacks to deep n eural networks with- out training substitute models. In Proceedings of the 10th ACM workshop on artificial intelligence and security , pages 15...
2017
-
[9]
Query-efficient hard-label black-box attack: An o ptimization-based approach
Minhao Cheng, Thong Le, Pin-Yu Chen, Huan Zhang, JinFeng Yi, and Cho- Jui Hsieh. Query-efficient hard-label black-box attack: An o ptimization-based approach. In International Conference on Learning Representations , 2019. 13
2019
-
[10]
Sign-opt: A query-efficient hard-label adver sarial attack
Minhao Cheng, Simranjit Singh, Patrick Chen, Pin-Yu Ch en, Sijia Liu, and Cho-Jui Hsieh. Sign-opt: A query-efficient hard-label adver sarial attack. arXiv preprint arXiv:1909.10773 , 2019. 3, 13
1909 arXiv
-
[11]
Explaining and har- nessing adversarial examples
Ian J Goodfellow, Jonathon Shlens, and Christian Szege dy. Explaining and har- nessing adversarial examples. arXiv preprint arXiv:1412.6572 , 2014. 2
2014 arXiv
-
[12]
Mamba: Linear-time sequence mode ling with selective state spaces
Albert Gu and Tri Dao. Mamba: Linear-time sequence mode ling with selective state spaces. arXiv preprint arXiv:2312.00752 , 2023. 11
2023 arXiv
-
[13]
D eep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. D eep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 11
2016
-
[14]
Densely connected convolutional networks
Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilia n Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 4700–4708, 2017. 11
2017
-
[15]
Batch normalization: Accelerating deep n etwork training by reduc- ing internal covariate shift
Sergey Ioffe. Batch normalization: Accelerating deep n etwork training by reduc- ing internal covariate shift. arXiv preprint arXiv:1502.03167 , 2015. 11
2015 arXiv
-
[16]
Reinforcement learning: A survey
Leslie Pack Kaelbling, Michael L Littman, and Andrew W M oore. Reinforcement learning: A survey. Journal of artificial intelligence research , 4:237–285, 1996. 7
1996
-
[17]
Robust d ecision-based black-box adversarial attack via coarse-to-fine random sea rch
Byeong Cheon Kim, Youngjoon Yu, and Yong Man Ro. Robust d ecision-based black-box adversarial attack via coarse-to-fine random sea rch. 2021 IEEE Inter- national Conference on Image Processing (ICIP) , pages 3048–3052, 2021. 2
2021
-
[18]
Big transfer (bit): Genera l visual representation learning
Alexander Kolesnikov, Lucas Beyer, Xiaohua Zhai, Joan Puigcerver, Jessica Yung, Sylvain Gelly, and Neil Houlsby. Big transfer (bit): Genera l visual representation learning. In Computer Vision–ECCV 2020: 16th European Conference, Glas gow, UK, August 23–28, 2020, Proceedings, ...
2020
-
[19]
Adve rsarial examples in the physical world
Alexey Kurakin, Ian J Goodfellow, and Samy Bengio. Adve rsarial examples in the physical world. In Artificial intelligence safety and security , pages 99–112. Chapman and Hall/CRC, 2018. 2
2018
-
[20]
Pre-training of deep bidirec tional transformers for language understanding
JDMCK Lee and K Toutanova. Pre-training of deep bidirec tional transformers for language understanding. arXiv preprint arXiv:1810.04805 , 3(8), 2018. 11
2018 arXiv
-
[21]
Qeba: Query- efficient boundary-based blackbox attack
Huichen Li, Xiaojun Xu, Xiaolu Zhang, Shuang Yang, and B o Li. Qeba: Query- efficient boundary-based blackbox attack. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition , pages 1221–1230, 2020. 3, 8, 12, 13
2020
-
[22]
M l attack models: Adversarial attacks and data poisoning attacks
Jing Lin, Long Dang, Mohamed Rahouti, and Kaiqi Xiong. M l attack models: Adversarial attacks and data poisoning attacks. arXiv preprint arXiv:2112.02797 ,
-
[23]
A geometry- inspired decision-based attack
Yujia Liu, Seyed-Mohsen Moosavi-Dezfooli, and Pascal Frossard. A geometry- inspired decision-based attack. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 4890–4898, 2019. 3
2019
-
[24]
Back in black: A comparative evaluation of recent state-of-the- art black-box attacks
Kaleel Mahmood, Rigel Mahmood, Ethan Rathbun, and Mart en van Dijk. Back in black: A comparative evaluation of recent state-of-the- art black-box attacks. IEEE Access, 10:998–1019, 2021. 2
2021
-
[25]
On t he robustness of vision transformers to adversarial examples
Kaleel Mahmood, Rigel Mahmood, and Marten Van Dijk. On t he robustness of vision transformers to adversarial examples. In Proceedings of the IEEE/CVF international conference on computer vision , pages 7838–7847, 2021. 2
2021
-
[26]
Surfre e: a fast surrogate- free black-box attack
Thibault Maho, Teddy Furon, and Erwan Le Merrer. Surfre e: a fast surrogate- free black-box attack. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10430–10439, 2021. 3, 8, 13
2021
-
[27]
Convergence of q-learning: A simple p roof
Francisco S Melo. Convergence of q-learning: A simple p roof. Institute Of Systems and Robotics, Tech. Rep , pages 1–4, 2001. 7
2001
-
[28]
Human-level control through deep reinfor cement learning
Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Andr ei A Rusu, Joel Veness, Marc G Bellemare, Alex Graves, Martin Riedmiller, Andreas K Fidjeland, Georg Ostrovski, et al. Human-level control through deep reinfor cement learning. nature, 518(7540):529–533, 2015. 7
2015
-
[29]
Practical black-box attacks agai nst machine learning
Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, S omesh Jha, Z Berkay Ce- lik, and Ananthram Swami. Practical black-box attacks agai nst machine learning. In Proceedings of the 2017 ACM on Asia conference on computer an d communi- cations security, pages 506–519, 2017. 2
2017
-
[30]
Geoda: a geometric framework for black-box adversarial att acks
Ali Rahmati, Seyed-Mohsen Moosavi-Dezfooli, Pascal F rossard, and Huaiyu Dai. Geoda: a geometric framework for black-box adversarial att acks. In Proceedings of the IEEE/CVF conference on computer vision and pattern re cognition, pages 8446–8455, 2020. 3, 8, 13
2020
-
[31]
Cgba: curvature- aware geometric black-box attack
Md Farhamdur Reza, Ali Rahmati, Tianfu Wu, and Huaiyu Da i. Cgba: curvature- aware geometric black-box attack. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 124–133, 2023. 3, 4
2023
-
[32]
Decision- based query efficient adversarial attack via adaptive bounda ry learning
Meng Shen, Changyue Li, Hao Yu, Qi Li, Liehuang Zhu, and K e Xu. Decision- based query efficient adversarial attack via adaptive bounda ry learning. IEEE Transactions on Dependable and Secure Computing , 2023. 3, 4
2023
-
[33]
D ecision-based black- box attack against vision transformers via patch-wise adve rsarial removal
Yucheng Shi, Yahong Han, Yu-an Tan, and Xiaohui Kuang. D ecision-based black- box attack against vision transformers via patch-wise adve rsarial removal. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh , editors, Advances in Neural Information Processing S...
2022
-
[34]
Very deep convolutional networks for l arge-scale image recog- nition
Karen Simonyan. Very deep convolutional networks for l arge-scale image recog- nition. arXiv preprint arXiv:1409.1556 , 2014. 11
2014 arXiv
-
[35]
Bounceat tack: A query-efficient decision-based adversarial attack by bouncing into the wil d
Jie Wan, Jianhao Fu, Lijin Wang, and Ziqi Yang. Bounceat tack: A query-efficient decision-based adversarial attack by bouncing into the wil d. In 2024 IEEE Sym- posium on Security and Privacy (SP) , pages 1270–1286. IEEE, 2024. 3, 4
2024
-
[36]
Boost- ing adversarial transferability through enhanced momentu m
Xiaosen Wang, Jiadong Lin, Han Hu, Jingdong Wang, and Ku n He. Boost- ing adversarial transferability through enhanced momentu m. arXiv preprint arXiv:2103.10609, 2021. 8
2021 arXiv
-
[38]
Better diffusion models further improve adversarial traini ng
Zekai Wang, Tianyu Pang, Chao Du, Min Lin, Weiwei Liu, an d Shuicheng Yan. Better diffusion models further improve adversarial traini ng. In International Conference on Machine Learning , pages 36246–36263. PMLR, 2023. 12
2023
-
[39]
Q-learning
Christopher JCH Watkins and Peter Dayan. Q-learning. Machine learning, 8:279– 292, 1992. 7, 8, 11
1992
-
[40]
Learning fr om delayed rewards
Christopher John Cornish Hellaby Watkins. Learning fr om delayed rewards. 1989. 7
1989
-
[41]
Qe-dba: Q uery-efficient decision-based adversarial attacks via bayesian optimiza tion
Zhuosheng Zhang, Noor Ahmed, and Shucheng Yu. Qe-dba: Q uery-efficient decision-based adversarial attacks via bayesian optimiza tion. In 2024 Inter- national Conference on Computing, Networking and Communic ations (ICNC) , pages 783–788. IEEE, 2024. 3, 4
2024
-
[42]
Vision mamba: Efficient visual representation lea rning with bidirec- tional state space model
Lianghui Zhu, Bencheng Liao, Qian Zhang, Xinlong Wang, Wenyu Liu, and Xing- gang Wang. Vision mamba: Efficient visual representation lea rning with bidirec- tional state space model. arXiv preprint arXiv:2401.09417 , 2024. 11, 12 19
2024 arXiv
Reviewed August 12, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.