pith. sign in

arxiv: 2505.16714 · v2 · submitted 2025-05-22 · 🪐 quant-ph · cs.LG

Experimental robustness benchmarking of quantum neural networks on a superconducting quantum processor

Pith reviewed 2026-05-22 13:32 UTC · model grok-4.3

classification 🪐 quant-ph cs.LG
keywords quantum neural networksadversarial robustnesssuperconducting quantum processorquantum machine learningadversarial attacksrobustness boundsfidelity
0
0 comments X

The pith

Quantum neural networks on superconducting hardware show stronger resistance to adversarial attacks than classical networks.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper runs the first systematic experimental test of how 20-qubit quantum neural network classifiers on a superconducting processor hold up against deliberate input changes designed to produce wrong answers. It introduces a tailored attack algorithm that quantifies this robustness and compares it against theoretical limits based on state fidelity. The measurements indicate that QNNs outperform classical networks, with the difference traced to the natural noise present in quantum hardware, and that training the models against such attacks further improves their stability by smoothing input gradients. The observed robustness upper bound sits only 0.003 away from the predicted lower bound, confirming both the attack's power and the accuracy of the theoretical description.

Core claim

Through controlled experiments the authors show that QNN classifiers possess superior adversarial robustness relative to classical neural networks, an advantage they link directly to the inherent quantum noise of the processor, while the empirical upper bound on robustness extracted from attack trials deviates by only 3 times 10 to the minus 3 from the theoretical lower bound set by fidelity.

What carries the argument

An efficient adversarial attack algorithm specialized for QNNs that produces quantitative robustness measures and allows direct comparison to fidelity-derived theoretical bounds.

If this is right

  • Adversarial training reduces QNN sensitivity to targeted perturbations by regularizing input gradients.
  • The close agreement between measured and theoretical bounds validates both the attack algorithm and the fidelity-based analysis.
  • The benchmarking approach supplies a concrete method for testing and improving security in other quantum machine learning models.
  • QNNs may enable more reliable deployment in settings where an adversary can modify inputs.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If quantum noise is the source of the advantage, then future error-corrected or larger-scale QNNs could lose part of this robustness benefit.
  • The same noise-driven effect might appear in quantum algorithms other than classification tasks.
  • Controlled simulations that isolate noise from other hardware differences could test the attribution directly.

Load-bearing premise

The robustness advantage observed in QNNs stems from inherent quantum noise rather than from differences in model size, training details, or other implementation choices.

What would settle it

Run the same attack suite on classical networks that have been given added noise levels matched to those in the superconducting QNN and check whether the robustness gap disappears.

Figures

Figures reproduced from arXiv: 2505.16714 by Athanasios V. Vasilakos, Guo-Ping Guo, Hai-Feng Zhang, Hao-Ran Tao, Huan-Yu Liu, Ji Guan, Lei Du, Liang-Liang Guo, Peng Duan, Peng Wang, Ren-Ze Zhao, Sheng Zhang, Tian-Le Wang, Wei-Cheng Kong, Xiao-Yan Yang, Yang Yang, Yu-Chun Wu, Ze-An Zhao, Zhao-Yun Chen, Zhi-Long Jia.

Figure 1
Figure 1. Figure 1: FIG. 1 [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: FIG. 2 [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: FIG. 3 [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: FIG. 4 [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: FIG. 5 [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
read the original abstract

Quantum machine learning (QML) models, like their classical counterparts, are vulnerable to adversarial attacks, hindering their secure deployment. Here, we report the first systematic experimental robustness benchmark for 20-qubit quantum neural network (QNN) classifiers executed on a superconducting processor. Our benchmarking framework features an efficient adversarial attack algorithm designed for QNNs, enabling quantitative characterization of adversarial robustness and robustness bounds. From our analysis, we verify that adversarial training reduces sensitivity to targeted perturbations by regularizing input gradients, significantly enhancing QNN's robustness. Additionally, our analysis reveals that QNNs exhibit superior adversarial robustness compared to classical neural networks, an advantage attributed to inherent quantum noise. Furthermore, the empirical upper bound extracted from our attack experiments shows a minimal deviation ($3 \times 10^{-3}$) from the theoretical lower bound, providing strong experimental confirmation of the attack's effectiveness and the tightness of fidelity-based robustness bounds. This work establishes a critical experimental framework for assessing and improving quantum adversarial robustness, paving the way for secure and reliable QML applications.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The manuscript reports the first systematic experimental robustness benchmark of 20-qubit QNN classifiers executed on a superconducting processor. It introduces an efficient adversarial attack algorithm tailored to QNNs, shows that adversarial training reduces sensitivity via input-gradient regularization, claims that QNNs exhibit superior adversarial robustness relative to classical NNs due to inherent quantum noise, and finds that the empirical upper bound extracted from attack experiments deviates by only 3×10^{-3} from the theoretical fidelity-based lower bound.

Significance. If the central experimental results hold, the work is significant because it supplies the first hardware-grounded benchmarking framework for quantum adversarial robustness, including concrete 20-qubit runs and a near-tight empirical-theoretical bound agreement. These outcomes directly support the utility of the proposed attack algorithm and the value of gradient-regularized adversarial training for QNNs on near-term devices.

major comments (1)
  1. [Abstract] Abstract: the claim that QNNs exhibit superior adversarial robustness 'attributed to inherent quantum noise' is load-bearing for the mechanistic interpretation yet rests on a comparison between the 20-qubit hardware QNN and a classical NN without reported controls that isolate hardware noise from differences in model capacity, architecture, optimizer, or gradient-regularization strength. A controlled ablation (or equivalent classical noise injection) is required to substantiate the causal attribution.
minor comments (2)
  1. [Methods] Methods section: provide explicit details on data-exclusion criteria, error-bar computation, and the precise classical baseline architecture and hyper-parameters to allow independent verification of the reported robustness gap.
  2. Figure captions and text: ensure all reported numerical values (e.g., the 3×10^{-3} deviation) are accompanied by the corresponding statistical uncertainty or number of experimental shots.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive feedback and the opportunity to improve the manuscript. We address the major comment point by point below and will incorporate revisions to strengthen the causal interpretation of our results.

read point-by-point responses
  1. Referee: [Abstract] Abstract: the claim that QNNs exhibit superior adversarial robustness 'attributed to inherent quantum noise' is load-bearing for the mechanistic interpretation yet rests on a comparison between the 20-qubit hardware QNN and a classical NN without reported controls that isolate hardware noise from differences in model capacity, architecture, optimizer, or gradient-regularization strength. A controlled ablation (or equivalent classical noise injection) is required to substantiate the causal attribution.

    Authors: We acknowledge that the current experimental comparison, while demonstrating superior robustness of the hardware QNN, does not include explicit controls to fully isolate hardware noise from other potential confounding factors such as model capacity or architecture differences. To address this, we will add a new subsection in the revised manuscript presenting a controlled ablation: specifically, we will include simulations of classical neural networks with injected noise calibrated to match the measured decoherence, gate infidelity, and readout error rates from the superconducting processor. We will also ensure the classical baseline uses comparable capacity and the same adversarial training protocol. This addition will provide stronger evidence for the role of inherent quantum noise while transparently discussing remaining limitations in the attribution. revision: yes

Circularity Check

0 steps flagged

No significant circularity detected in experimental benchmarking results

full rationale

The paper reports hardware experiments measuring adversarial robustness of a 20-qubit QNN on a superconducting processor, including an attack algorithm and comparison of empirical upper bounds to a theoretical lower bound (deviation of 3e-3). These are direct measurements and bound checks rather than any claimed derivation chain. No equations or steps reduce by construction to fitted inputs, self-citations, or ansatzes; the central claims rest on experimental data and an independent theoretical bound rather than tautological redefinitions or load-bearing self-references. The attribution of robustness to quantum noise is an interpretive statement without supporting derivation that could be circular.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The work is primarily experimental. No new mathematical axioms or invented physical entities are introduced. The main background assumptions are standard quantum mechanics and the validity of the fidelity-based robustness bound from prior theory. No free parameters are fitted to produce the headline claims; the 3e-3 deviation is a measured quantity.

axioms (1)
  • domain assumption Fidelity-based robustness bounds derived in prior theoretical work remain valid when applied to real superconducting hardware.
    The paper uses the closeness of empirical upper bound to theoretical lower bound as confirmation; this assumes the theoretical bound still applies under device noise and gate errors.

pith-pipeline@v0.9.0 · 5788 in / 1519 out tokens · 30440 ms · 2026-05-22T13:32:13.137220+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

49 extracted references · 49 canonical work pages · 4 internal anchors

  1. [1]

    Vaswani, N

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. u. Kaiser, and I. Polosukhin, Attention is all you need, inAdvances in Neural Information Processing Systems, V ol. 30 (Curran Associates, Inc., 2017) pp. 5998–6008

  2. [2]

    P. S. Chib and P. Singh, Recent advancements in end-to-end au- tonomous driving using deep learning: A survey, IEEE Trans- actions on Intelligent Vehicles9, 103 (2023)

  3. [3]

    Towards Deep Learning Models Resistant to Adversarial Attacks

    A. Madry, A. Makelov, L. Schmidt, D. Tsipras, and A. Vladu, Towards deep learning models resistant to adversarial attacks, arXiv:1706.06083 (2019)

  4. [4]

    Zhong and W

    Y . Zhong and W. Deng, Towards transferable adversarial attack against deep face recognition, IEEE Transactions on Informa- tion Forensics and Security16, 1452 (2021)

  5. [5]

    Y . Cao, Y . Zhu, D. Wang, S. Wen, M. Xue, J. Lu, and H. Ge, Rethinking the threat and accessibility of adversarial attacks against face recognition systems, arXiv:2407.08514 (2024)

  6. [6]

    Shafique, M

    M. Shafique, M. Naseer, T. Theocharides, C. Kyrkou, O. Mutlu, L. Orosa, and J. Choi, Robust machine learning systems: Chal- lenges, current trends, perspectives, and the road ahead, IEEE Design & Test37, 30 (2020)

  7. [7]

    Biamonte, P

    J. Biamonte, P. Wittek, N. Pancotti, P. Rebentrost, N. Wiebe, and S. Lloyd, Quantum machine learning, Nature549, 195 (2017)

  8. [8]

    Cerezo, G

    M. Cerezo, G. Verdon, H.-Y . Huang, L. Cincio, and P. J. Coles, Challenges and opportunities in quantum machine learning, Na- ture computational science2, 567 (2022)

  9. [9]

    Lloyd, M

    S. Lloyd, M. Mohseni, and P. Rebentrost, Quantum principal component analysis, Nature physics10, 631 (2014)

  10. [10]

    Rebentrost, M

    P. Rebentrost, M. Mohseni, and S. Lloyd, Quantum support vec- tor machine for big data classification, Phys. Rev. Lett.113, 130503 (2014)

  11. [11]

    Dunjko, J

    V . Dunjko, J. M. Taylor, and H. J. Briegel, Quantum-enhanced machine learning, Phys. Rev. Lett.117, 130501 (2016)

  12. [12]

    Huang, M

    H.-Y . Huang, M. Broughton, J. Cotler, S. Chen, J. Li, M. Mohseni, H. Neven, R. Babbush, R. Kueng, J. Preskill, and J. R. McClean, Quantum advantage in learning from experi- ments, Science376, 1182 (2022)

  13. [13]

    Havl´ıˇcek, A

    V . Havl´ıˇcek, A. D. C´orcoles, K. Temme, A. W. Harrow, A. Kan- dala, J. M. Chow, and J. M. Gambetta, Supervised learning with quantum-enhanced feature spaces, Nature567, 209 (2019)

  14. [14]

    Herrmann, S

    J. Herrmann, S. M. Llima, A. Remm, P. Zapletal, N. A. McMa- hon, C. Scarato, F. Swiadek, C. K. Andersen, C. Hellings, S. Krinner, N. Lacroix, S. Lazar, M. Kerschbaum, D. C. Zanuz, G. J. Norris, M. J. Hartmann, A. Wallraff, and C. Eichler, Re- alizing quantum convolutional neural networks on a supercon- ducting quantum processor to recognize quantum phases...

  15. [15]

    M. Gong, H. L. Huang, S. Wang, C. Guo, S. Li, Y . Wu, Q. Zhu, Y . Zhao, S. Guo, H. Qian, Y . Ye, C. Zha, F. Chen, C. Ying, J. Yu, D. Fan, D. Wu, H. Su, H. Deng, H. Rong, K. Zhang, S. Cao, J. Lin, Y . Xu, L. Sun, C. Guo, N. Li, F. Liang, A. Sakurai, K. Nemoto, W. J. Munro, Y . H. Huo, C. Y . Lu, C. Z. Peng, X. Zhu, and J. W. Pan, Quantum neuronal sensing o...

  16. [16]

    Tacchino, C

    F. Tacchino, C. Macchiavello, D. Gerace, and D. Bajoni, An artificial neuron implemented on an actual quantum processor, npj Quantum Information5, 26 (2019)

  17. [17]

    Huang, Y

    H.-L. Huang, Y . Du, M. Gong, Y . Zhao, Y . Wu, C. Wang, S. Li, F. Liang, J. Lin, Y . Xu, R. Yang, T. Liu, M.-H. Hsieh, H. Deng, H. Rong, C.-Z. Peng, C.-Y . Lu, Y .-A. Chen, D. Tao, X. Zhu, and J.-W. Pan, Experimental quantum generative adversarial networks for image generation, Phys. Rev. Appl.16, 024051 (2021)

  18. [18]

    Huang, Z.-A

    K. Huang, Z.-A. Wang, C. Song, K. Xu, H. Li, Z. Wang, Q. Guo, Z. Song, Z.-B. Liu, D. Zheng,et al., Quantum genera- tive adversarial networks with multiple superconducting qubits, npj Quantum Information7, 165 (2021)

  19. [19]

    Zhang, Z

    C. Zhang, Z. Lu, L. Zhao, S. Xu, W. Li, K. Wang, J. Chen, Y . Wu, F. Jin, X. Zhu,et al., Quantum continual learning on a programmable superconducting processor, arXiv:2409.09729 (2024)

  20. [20]

    J. Chen, Y . Wu, Z. Yang, S. Xu, X. Ye, D. Li, K. Wang, C. Zhang, F. Jin, X. Zhu,et al., Quantum ensemble learning with a programmable superconducting processor, npj Quantum Information11, 83 (2025)

  21. [21]

    Lu, L.-M

    S. Lu, L.-M. Duan, and D.-L. Deng, Quantum adversarial ma- chine learning, Phys. Rev. Res.2, 033212 (2020)

  22. [22]

    Liu and P

    N. Liu and P. Wittek, Vulnerability of quantum classification to adversarial perturbations, Phys. Rev. A101, 062331 (2020)

  23. [23]

    Franco, A

    N. Franco, A. Sakhnenko, L. Stolpmann, D. Thuerck, F. Petsch, A. R¨ull, and J. M. Lorenz, Predominant aspects on security for quantum machine learning: Literature review, in2024 IEEE In- ternational Conference on Quantum Computing and Engineer- ing (QCE), V ol. 01 (2024) pp. 1467–1477

  24. [24]

    J. Guan, W. Fang, and M. Ying, Robustness verification of quantum classifiers, inComputer Aided Verification(Springer International Publishing, Cham, 2021) pp. 151–174

  25. [25]

    Y . Lin, J. Guan, W. Fang, M. Ying, and Z. Su, Veriqr: A ro- bustness verification tool for quantum machine learning mod- els, arXiv:2407.13533 (2024)

  26. [26]

    M. T. West, S. M. Erfani, C. Leckie, M. Sevior, L. C. L. Hol- lenberg, and M. Usman, Benchmarking adversarially robust quantum machine learning at scale, Phys. Rev. Res.5, 023186 (2023)

  27. [27]

    W. Ren, W. Li, S. Xu, K. Wang, W. Jiang, F. Jin, X. Zhu, J. Chen, Z. Song, P. Zhang, H. Dong, X. Zhang, J. Deng, Y . Gao, C. Zhang, Y . Wu, B. Zhang, Q. Guo, H. Li, Z. Wang, J. Biamonte, C. Song, D. L. Deng, and H. Wang, Experimen- tal quantum adversarial learning with programmable supercon- ducting qubits, Nature Computational Science2, 711 (2022)

  28. [28]

    Benedetti, E

    M. Benedetti, E. Lloyd, S. Sack, and M. Fiorentini, Parame- terized quantum circuits as machine learning models, Quantum Science and Technology4, 043001 (2019)

  29. [29]

    Cerezo, A

    M. Cerezo, A. Arrasmith, R. Babbush, S. C. Benjamin, S. Endo, K. Fujii, J. R. McClean, K. Mitarai, X. Yuan, L. Cincio, and P. J. Coles, Variational quantum algorithms, Nature Reviews Physics3, 625 (2021)

  30. [30]

    Bharti, A

    K. Bharti, A. Cervera-Lierta, T. H. Kyaw, T. Haug, S. Alperin- Lea, A. Anand, M. Degroote, H. Heimonen, J. S. Kottmann, 10 T. Menke, W.-K. Mok, S. Sim, L.-C. Kwek, and A. Aspuru- Guzik, Noisy intermediate-scale quantum algorithms, Rev. Mod. Phys.94, 015004 (2022)

  31. [31]

    I. J. Goodfellow, J. Shlens, and C. Szegedy, Explaining and har- nessing adversarial examples, arXiv:1412.6572 (2014)

  32. [32]

    T.-W. Weng, H. Zhang, P.-Y . Chen, J. Yi, D. Su, Y . Gao, C.-J. Hsieh, and L. Daniel, Evaluating the robustness of neural net- works: An extreme value theory approach, arXiv:1801.10578 (2018)

  33. [33]

    C.-Y . Ko, Z. Lyu, L. Weng, L. Daniel, N. Wong, and D. Lin, Parametric noise injection: Trainable randomness to improve deep neural network robustness against adversarial attack, inIn- ternational Conference on Machine Learning(2019) pp. 3468– 3477

  34. [34]

    Z. He, A. S. Rakin, and D. Fan, Parametric noise injec- tion: Trainable randomness to improve deep neural network robustness against adversarial attack, inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition (CVPR)(2019) pp. 588–597

  35. [35]

    A. Liu, X. Liu, H. Yu, C. Zhang, Q. Liu, and D. Tao, Training robust deep neural networks via adversarial noise propagation, Trans. Img. Proc.30, 5769–5781 (2021)

  36. [36]

    N. Ye, L. Cao, L. Yang, Z. Zhang, Z. Fang, Q. Gu, and G.-Z. Yang, Improving the robustness of analog deep neural networks through a bayes-optimized noise injection approach, Communi- cations Engineering2, 25 (2023)

  37. [37]

    Gong and D.-L

    W. Gong and D.-L. Deng, Universal adversarial examples and perturbations for quantum classifiers, National Science Review 9, nwab130 (2021)

  38. [38]

    Papernot, P

    N. Papernot, P. McDaniel, X. Wu, S. Jha, and A. Swami, Dis- tillation as a defense to adversarial perturbations against deep neural networks, in2016 IEEE Symposium on Security and Pri- vacy (SP)(2016) pp. 582–597

  39. [39]

    A. S. Ros and F. Doshi-Velez, Improving the adversar- ial robustness and interpretability of deep neural networks by regularizing their input gradients, inProceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, AAAI’18/IAAI’18/EAAI’18 (AAAI Press, 2018) pp. 1660– 1669

  40. [40]

    P ´erez-Salinas, A

    A. P ´erez-Salinas, A. Cervera-Lierta, E. Gil-Fuster, and J. I. Latorre, Data re-uploading for a universal quantum classifier, Quantum4, 226 (2020)

  41. [41]

    T. Haug, C. N. Self, and M. S. Kim, Quantum machine learn- ing of large datasets using randomized measurements, Machine Learning: Science and Technology4, 015005 (2023)

  42. [43]

    D. C. McKay, C. J. Wood, S. Sheldon, J. M. Chow, and J. M. Gambetta, Efficientzgates for quantum computing, Phys. Rev. A96, 022330 (2017)

  43. [44]

    Mitarai, M

    K. Mitarai, M. Negoro, M. Kitagawa, and K. Fujii, Quantum circuit learning, Phys. Rev. A98, 032309 (2018)

  44. [45]

    Schuld, V

    M. Schuld, V . Bergholm, C. Gogolin, J. Izaac, and N. Killoran, Evaluating analytic gradients on quantum hardware, Phys. Rev. A99, 032331 (2019)

  45. [46]

    D. P. Kingma and J. Ba, Adam: A method for stochastic opti- mization, arXiv:1412.6980 (2014)

  46. [47]

    C. A. Fuchs and C. M. Caves, Ensemble-dependent bounds for accessible information in quantum mechanics, Phys. Rev. Lett. 73, 3047 (1994)

  47. [48]

    M. A. Nielsen, A simple formula for the average gate fidelity of a quantum dynamical operation, Physics Letters A303, 249 (2002)

  48. [49]

    Experimental robustness benchmarking of quantum neural networks on a superconducting quantum processor

    B. Nachman, M. Urbanek, W. A. de Jong, and C. W. Bauer, Unfolding quantum computer readout noise, npj Quantum In- formation6, 84 (2020). S1 Supplementary materials for “Experimental robustness benchmarking of quantum neural networks on a superconducting quantum processor” CONTENTS I. Quantum neural network classifier S1 II. Training algorithm S3 III. Mask...

  49. [50]

    low-pass filter

    = (1−ξ) l 1−z 2 0 1−(1−ξ) 2lz2 0| {z } :=C(ξ,l,z 0) Sideal.(S37) whereS ideal is the sensitivity in the noiseless condition. The above equation provides the exact first-order ratio factorC(ξ, l, z0), which scales the noiseless sensitivity to the noisy situation. S11 Letu= (1−ξ) l,u∈[0,1]. We now prove thatS noisy ≤S ideal is equivalent to proving C(ξ, l, ...