REVIEW 3 major objections 5 minor 26 references
CAPAA: Classifier-Agnostic Projector-Based Adversarial Attack
T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read CAPAA projects a single light pattern that fools several image classifiers at once and keeps working when the camera angle changes.
desk verdict First classifier-agnostic projector attack with real physical experiments, but the pose-robustness claim outruns the evidence and the uniform success-rate claim is undercut by the paper's own Table I. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine of the method is a weighted combination of three parts: (1) a classifier-agnostic adversarial loss that sums per-classifier logit losses, or temperature-controlled LogSoftmax losses for targeted attacks; (2) a perceptual stealthiness loss using CIEDE2000 color distance; and (3) a perturbation attention map A, computed once from the weighted sum of Grad-CAM++ class activation maps on the clean reference image, which multiplies the loss gradient elementwise so perturbations concentrate on decision-critical object regions. The optimization alternates: when the simulated attack is not yet confident enough or the perturbation is too small, it minimizes the adversarial loss; otherwise it minimizes the stealthiness loss, with both updates scaled by A. The project-and-capture process is modeled by a learned network PCNet so the whole loop stays differentiable.
What would settle it
Capture a CAPAA projection while rotating or zooming the camera so that the region highlighted by the reference Grad-CAM++ map is occluded or leaves the frame, and record whether the attack success rate collapses; if the success rate stays high, the attention-weighting premise is not the load-bearing mechanism, and if it collapses, the premise is confirmed.
Extended reading notes
Core claim
CAPAA's central claim is that a classifier-agnostic adversarial projection can be produced by optimizing a single loss that aggregates the adversarial and stealthiness gradients of several classifiers, with the gradient multiplied pointwise by a perturbation attention map built from the classifiers' Grad-CAM++ activations on the clean reference image. The temperature-controlled LogSoftmax term for targeted attacks pushes the target logit well above all others, so that a real-world projection still lands on the intended class rather than a near neighbor. The paper reports that on 10 setups with 7 camera poses each, CAPAA reaches a top-1 average success rate of 82.02% for untargeted attacks, matching the no-attention variant but with smaller perceptual color distance and higher SSIM, and outperforms the single-classifier SPAA baseline, which reaches 64.68%.
Load-bearing premise
The attack decides where to place its visible changes by looking at which parts of the object are highlighted in a single clean reference photo, and it assumes those same parts remain the decisive ones after the pattern is projected and after the camera moves.
Editorial extensions
If this is right
- A single projector pattern can be designed to attack an ensemble of classifiers at once, so the cost of protecting a scene does not grow with the number of recognition systems guarding it.
- Because perturbations are concentrated on the object region the classifiers attend to, the attack is less likely to be spoiled when a camera shift occludes background parts of the projected pattern.
- The success rate under pose changes is bounded by the attention map: regions the reference Grad-CAM++ misses will not receive perturbation, so objects whose discriminative parts move with pose will be attacked less reliably.
- Perceptual color distance rather than pixel norms is the operative stealthiness measure, so the same framework can be judged by how visible the change is to a person, not just by its magnitude in image space.
Reading between the lines
- A natural next step the paper does not take is to compute the attention map as an average over multiple camera poses instead of a single reference pose; that would directly test whether the CAM premise, not the optimization, is what limits pose reliability.
- If CAPAA transfers to unseen architectures as the supplementary ViT and unseen-CNN experiments suggest, the same pattern could be used as a privacy shield against future, unknown classifiers, but the authors' own numbers show transfer to newer ViT variants is still weak.
- The alternating optimization with thresholds pthr and dthr can be read as a principled way to trace a stealthiness-versus-success tradeoff frontier; a practitioner could fix dthr to a budget and read off the achievable success rate.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes CAPAA, a projector-based physical adversarial attack method intended to fool multiple image classifiers simultaneously and to remain effective under camera pose changes. The method reuses the PCNet neural simulator from SPAA to model the project-and-capture process, introduces a classifier-agnostic adversarial loss that aggregates per-classifier adversarial and stealthiness gradients, and weights these gradients with a Grad-CAM++-based attention map computed on the clean reference image. Experiments on 10 object setups, 7 camera poses, and three classifiers compare CAPAA with SPAA and two ablations (CAPAA without attention and CAPAA classifier-specific), reporting stealthiness metrics and attack success rates. The paper claims that CAPAA achieves both a higher attack success rate and greater stealthiness than baselines.
Significance. If the claims hold, CAPAA would be a useful practical tool for privacy protection and adversarial robustness evaluation, and it appears to be the first classifier-agnostic projector-based attack. The paper's strengths include a real physical testbed with multiple objects, poses, and classifiers; the release of code; and clean ablations that isolate the effect of attention weighting and multi-classifier aggregation. The qualitative results in Figures 4 and 5 indicate that attention-based perturbation placement can avoid occlusions and field-of-view changes. However, the headline quantitative claim is not uniformly supported by Table I, and pose robustness is only demonstrated through pose-averaged metrics, so the significance is currently conditional on additional per-pose and statistical evidence.
major comments (3)
- [Section III.B, Table I] The central claim that CAPAA achieves a higher attack success rate is not supported by the reported numbers: CAPAA and CAPAA w/o attention both have an average attack success rate of 82.02%, and CAPAA w/o attention is higher at d_thr=4 (87.14% vs. 84.76%) and at d_thr=5 (87.62% vs. 86.67%). No error bars, confidence intervals, or per-setup results are provided, so the observed differences cannot be assessed statistically. Please provide per-setup success rates and repeated-trial statistics, or revise the claim to state that CAPAA matches the success rate of the no-attention ablation while improving stealthiness at low perturbation thresholds.
- [Section II.B, Eq. (8); Section IV, Limitations] The claimed robustness to camera pose changes is not directly evidenced. The attention map A is computed once from the clean reference image I_{x0,gamma0}, PCNet is trained only at gamma0 (Eq. 5), and Algorithm 1 only simulates the capture at gamma0; the paper's limitation section concedes that CAPAA 'does not aggregate attack loss gradients from multiple camera poses.' The evaluation metric in Eq. (9) averages over all 7 poses, which can hide systematic failures at poses where the highlighted regions are occluded or leave the field of view (as seen qualitatively in Fig. 5). Please report per-pose success rates (original, 15 degrees, 30 degrees, focal-length changes) for CAPAA and the baselines, and discuss how the static CAM prior behaves at low-overlap poses.
- [Section III.B, Fig. 6(b)] The sentence 'CAPAA and CAPAA (w/o attention) lead in performance for classifier-agnostic targeted attacks at the original camera pose, with CAPAA (w/o attention) tripling the success rate due to the three classifiers targeted' is ambiguous and appears to indicate that removing attention triples the targeted attack success rate. This would contradict the unqualified abstract claim that CAPAA achieves a higher attack success rate. Please clarify with numerical success rates for targeted attacks, and either justify why the attention mechanism is still beneficial despite this regression or restrict the headline claim to untargeted attacks.
minor comments (5)
- [Section III.B, first paragraph] The text says 'As shown in Table II' when referring to the untargeted attack quantitative results, but those results are in Table I of the main text; Table II in the supplementary reports ViT experiments.
- [Section II.B, Eq. (8)] The notation L_CAPAA is used in Eq. (8) but never defined; please define it explicitly as the combined adversarial and stealthiness objective, or replace it with the explicit sum of the two loss gradients.
- [Algorithm 1] The variable f_yt is used to decide the optimization branch, but the target class y_t is not defined for untargeted attacks; please clarify which probability is being thresholded in the untargeted case.
- [Section III.B, Table I discussion] The text mentions 'a marginal 0.001% difference' between CAPAA and CAPAA w/o attention, but Table I shows identical average success rates (82.02%); please correct this inconsistency.
- [Figure 6] The captions for Fig. 6 should state explicitly which poses are included in each panel (P=1 vs. P=7) and how the success-rate-versus-stealthiness curves are aggregated over classifiers and setups.
Circularity Check
No significant circularity: the optimization objective is not definitionally the evaluation quantity, PCNet is retrained and physically validated, and the only self-citation (SPAA) is a baseline and architectural inspiration, not load-bearing evidence.
full rationale
CAPAA is an empirical attack-optimization paper, not a derivation from first principles, and its claimed outputs are not definitionally identical to its inputs. The classifier-agnostic loss (Eqs. 6-7) directly optimizes the classifiers in the set, but success is measured on real projected-and-captured images (Eq. 9), so the multi-classifier claim is an experimental transfer result, not a tautology. The differentiable project-and-capture model PCNet (Eq. 5) is trained on real capture pairs and used only as an optimization proxy; the final numbers in Table I and Figs. 4-5 come from physical projection, so the simulator is not the test set. The attention map A is computed once from Grad-CAM++ on the clean reference image and used as a fixed gradient mask (Eq. 8); whether this improves pose robustness is an empirical hypothesis tested against CAPAA w/o attention and SPAA, not an assumption that defines the result. The only self-citation is SPAA [10], by the corresponding author, which supplies the PCNet idea and serves as a baseline; because PCNet is retrained and the attack is physically validated, this citation is not load-bearing in the required sense. The paper's own limitation statement, 'CAPAA is not pose-agnostic because it does not aggregate attack loss gradients from multiple camera poses,' narrows the pose-robustness claim but is a scope caveat, not a circular step. The observation that Table I shows equal pose-averaged success for CAPAA and CAPAA w/o attention (82.02%) and lower CAPAA success at dthr=4,5 is an evidence-quality concern for the attention mechanism, not a reduction of the conclusion to its premises.
Assumptions & free parameters
free parameters (6)
- PCNet parameters theta =
trained on M real projection-capture pairs at pose gamma0
- Temperature T for targeted LogSoftmax loss =
dynamically adjusted, schedule undisclosed
- Classifier weights omega_k =
not stated, presumably uniform
- PAM weights mu^(k) =
1/N per classifier
- Thresholds p_thr and d_thr =
p_thr=0.9, d_thr in {2,3,4,5}
- Gradient step sizes beta1, beta2 =
beta1=2, beta2=1
assumptions (3)
- domain assumption Differentiable PCNet faithfully approximates the physical project-and-capture process pi
- ad hoc to paper Grad-CAM++ map from the clean reference image remains decision-relevant under perturbation and pose changes
- domain assumption Multi-classifier logit sum produces a transferable single projector pattern
Cite this review
Pith. "Pith review of CAPAA: Classifier-Agnostic Projector-Based Adversarial Attack." pith.science (2026). https://pith.science/paper/L732GHUX
@misc{pith2026250600978,
author = {Pith},
title = {Pith review of: CAPAA: Classifier-Agnostic Projector-Based Adversarial Attack},
year = {2026},
howpublished = {\url{https://pith.science/paper/L732GHUX}},
note = {Machine review of arXiv:2506.00978}
}
read the original abstract
Projector-based adversarial attack aims to project carefully designed light patterns (i.e., adversarial projections) onto scenes to deceive deep image classifiers. It has potential applications in privacy protection and the development of more robust classifiers. However, existing approaches primarily focus on individual classifiers and fixed camera poses, often neglecting the complexities of multi-classifier systems and scenarios with varying camera poses. This limitation reduces their effectiveness when introducing new classifiers or camera poses. In this paper, we introduce Classifier-Agnostic Projector-Based Adversarial Attack (CAPAA) to address these issues. First, we develop a novel classifier-agnostic adversarial loss and optimization framework that aggregates adversarial and stealthiness loss gradients from multiple classifiers. Then, we propose an attention-based gradient weighting mechanism that concentrates perturbations on regions of high classification activation, thereby improving the robustness of adversarial projections when applied to scenes with varying camera poses. Our extensive experimental evaluations demonstrate that CAPAA achieves both a higher attack success rate and greater stealthiness compared to existing baselines. Codes are available at: https://github.com/ZhanLiQxQ/CAPAA.
Figures
Figures from the paper (3 more)
Reference graph
Works this paper leans on
-
[1]
" write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION new.ncblock write newline " " before.all 'output.state := FUNCTION new.nccont write " " before.all 'output.state := FUNCTION new.sentence output.state after.block = 'skip out...
-
[2]
11em plus .33em minus .07em 4000 4000 100 4000 4000 500 `\.=1000 = #1 \@IEEEnotcompsoconly \@IEEEcompsoconly #1 * [1] 0pt [0pt][0pt] #1 * [1] 0pt [0pt][0pt] #1 * \| ** #1 \@IEEEauthorblockNstyle \@IEEEcompsocnotconfonly \@IEEEauthorblockAstyle \@IEEEcompsocnotconfonly \@IEEEcompsocconfonly \@IEEEauthordefaulttextstyle \@IEEEcompsocnotconfonly \@IEEEauthor...
-
[3]
I. J. Goodfellow, J. Shlens, and C. Szegedy, ``Explaining and harnessing adversarial examples,'' ICLR, vol. abs/1412.6572, 2015
arXiv 2015
- [4]
-
[5]
P. Benz, C. Zhang, A. Karjauv, and I. S. Kweon, ``Universal adversarial training with class-wise perturbations,'' in ICME, 2021, pp. 1--6
work page 2021
-
[6]
J. Fang, Y. Jiang, C. Jiang, Z. L. Jiang, C. Liu, and S.-M. Yiu, ``State-of-the-art optical-based physical adversarial attacks for deep learning computer vision systems,'' ESWA, p. 123761, 2024
work page 2024
-
[7]
X. Wei, Y. Guo, and J. Yu, ``Adversarial sticker: A stealthy attack method in the physical world,'' TPAMI, vol. 45, pp. 2711--2725, 2021
work page 2021
-
[8]
J. Deng, W. Dong, R. Socher, L.-J. Li, L. Kai, and F.-F. Li, ``Imagenet: A large-scale hierarchical image database,'' in CVPR, 2009, pp. 248--255
work page 2009
Show all 26 references
-
[9]
Gnanasambandam, A
A. Gnanasambandam, A. M. Sherman, and S. H. Chan, ``Optical adversarial attack,'' ICCVW, pp. 92--101, 2021
2021
-
[10]
C. Hu, W. Shi, and L. Tian, ``Adversarial color projection: A projector-based physical-world attack to dnns,'' Image and Vision Computing, vol. 140, p. 104861, 2023
2023
-
[11]
H. Wei, H. Tang, X. Jia, Z. Wang, H. Yu, Z. Li, S. Satoh, L. Van Gool, and Z. Wang, ``Physical adversarial attack meets computer vision: A decade survey,'' TPAMI, vol. 46, no. 12, pp. 9797--9817, 2024
2024
-
[12]
Huang and H
B. Huang and H. Ling, ``Spaa: Stealthy projector-based adversarial attacks on deep image classifiers,'' in VR, 2022, pp. 534--542
2022
-
[13]
Y. Guo, X. Wang, P. Xiao, and X. Xu, ``An ensemble learning framework for convolutional neural network based on multiple classifiers,'' Soft Computing, vol. 24, no. 5, pp. 3727--3735, 2020
2020
-
[14]
M. R. Luo, G. Cui, and B. Rigg, ``The development of the CIE 2000 colour-difference formula: CIEDE2000 ,'' Color Research & Application, vol. 26, no. 5, pp. 340--350, 2001
2000
-
[15]
Z. Zhao, Z. Liu, and M. Larson, ``Towards large yet imperceptible adversarial image perturbations with perceptual color distance,'' in CVPR, 2020, pp. 1036--1045
2020
-
[16]
Chattopadhyay, A
A. Chattopadhyay, A. Sarkar, P. Howlader, and V. N. Balasubramanian, ``Grad-cam++: Generalized gradient-based visual explanations for deep convolutional networks,'' WACV, pp. 839--847, 2017
2017
-
[17]
Redmon, S
J. Redmon, S. Divvala, R. Girshick, and A. Farhadi, ``You only look once: Unified, real-time object detection,'' in CVPR, 2016
2016
-
[18]
K. He, X. Zhang, S. Ren, and J. Sun, ``Deep residual learning for image recognition,'' in CVPR, 2016, pp. 770--778
2016
-
[19]
Simonyan and A
K. Simonyan and A. Zisserman, ``Very deep convolutional networks for large-scale image recognition,'' in ICLR, 2015
2015
-
[20]
Szegedy, V
C. Szegedy, V. Vanhoucke, S. Ioffe, J. Shlens, and Z. Wojna, ``Rethinking the inception architecture for computer vision,'' in CVPR, 2016, pp. 2818--2826
2016
-
[21]
Dosovitskiy et al., ``An image is worth 16x16 words: Transformers for image recognition at scale,'' in ICLR, 2021
A. Dosovitskiy et al., ``An image is worth 16x16 words: Transformers for image recognition at scale,'' in ICLR, 2021
2021
-
[22]
Dosovitskiy et al., ``An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,'' in ICLR, 2021
A. Dosovitskiy et al., ``An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale,'' in ICLR, 2021
2021
-
[23]
Z. Liu, H. Mao, C.-Y. Wu, C. Feichtenhofer, T. Darrell, and S. Xie, ``A ConvNet for the 2020s,'' in CVPR, 2022, pp. 11966-11976
2022
-
[24]
Tan and Q
M. Tan and Q. V. Le, ``EfficientNet: Rethinking model scaling for convolutional neural networks,'' in ICML, 2019, pp. 10691-10700
2019
-
[25]
Howard, M
A. Howard, M. Sandler, G. Chu, L.-C. Chen, B. Chen, M. Tan, W. Wang, Y. Zhu, R. Pang, V. Vasudevan, Q. V. Le, and H. Adam, ``Searching for MobileNetV3,'' in ICCV, 2019, pp. 1314–1324
2019
-
[26]
Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, and B. Guo, ``Swin Transformer: Hierarchical vision transformer using shifted windows,'' in ICCV, 2021, pp. 9992-10002
2021
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.