pith. sign in

arxiv: 2507.12024 · v2 · submitted 2025-07-16 · 🪐 quant-ph

Benchmarking fault-tolerant quantum computing hardware via QLOPS

Pith reviewed 2026-05-19 04:54 UTC · model grok-4.3

classification 🪐 quant-ph
keywords fault-tolerant quantum computingQLOPSbenchmarkingquantum error correctiondecoder performanceRSA-2048resource estimation
0
0 comments X

The pith

QLOPS metric benchmarks fault-tolerant quantum hardware by factoring RSA-2048 resource needs.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces Quantum Logical Operations Per Second (QLOPS) as a benchmarking metric for fault-tolerant quantum computing schemes on hardware platforms. QLOPS folds together the code rates of error-correcting codes with the accuracy, speed, and delay of the decoders that correct errors during computation. The authors test the metric by estimating the resources required to factor a 2048-bit RSA number and show that the resulting number tracks what matters for running a real quantum algorithm. This single figure then points to the main hardware limits and allows different fault-tolerant designs to be ranked against one another. The work also supplies rough estimates of how many physical qubits and what decoder performance would be needed to reach useful quantum computation.

Core claim

QLOPS is defined as a scalar that multiplies the logical operation rate by the effective code rate while dividing by factors that capture decoder error rate, throughput, and latency. Resource analysis of Shor's algorithm for factoring RSA-2048 shows that this quantity directly measures the time and hardware overhead required to obtain the correct answer, thereby revealing which hardware features dominate the cost of fault-tolerant execution.

What carries the argument

QLOPS, the scalar formed by combining quantum error-correcting code rate with decoder accuracy, throughput, and latency to quantify the rate of useful logical operations.

If this is right

  • Hardware teams can rank proposed improvements by their expected effect on QLOPS rather than by isolated metrics such as physical error rate alone.
  • Designers obtain concrete targets for decoder speed and accuracy needed to reach a given logical operation rate.
  • Different error-correcting codes can be compared directly on the basis of the QLOPS they deliver for the same algorithm.
  • Rough timelines for when factoring or other algorithms become feasible follow from projecting how QLOPS scales with hardware progress.
  • Resource estimates for large algorithms become more realistic because they now include decoder overhead explicitly.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same QLOPS framework could be applied to other algorithms such as quantum simulation or optimization to check whether the bottlenecks identified for factoring are algorithm-specific.
  • Hardware roadmaps might shift emphasis toward faster classical control systems once QLOPS shows that decoder latency often limits overall performance.
  • Public benchmarks using QLOPS could let experimental groups publish comparable figures even when they use different error-correcting codes.

Load-bearing premise

Combining code rate, decoder accuracy, throughput, and latency into one scalar will reliably identify the dominant bottlenecks across different fault-tolerant schemes and hardware platforms.

What would settle it

Two FTQC schemes with different QLOPS values are run on the same hardware; if the lower-QLOPS scheme finishes RSA-2048 factoring faster or with fewer physical resources, the metric does not reflect practical requirements.

Figures

Figures reproduced from arXiv: 2507.12024 by Fang Zhang, Jianxin Chen, Linghang Kong.

Figure 1
Figure 1. Figure 1: Exponential fit of logical error rate vs code [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The achievable values of QLOPS density at [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 7
Figure 7. Figure 7: For the QLOPS estimation, we simply ig￾nore the reaction time and modify Eq. (2) into Q = k × 1/tSEC (assuming 1 syndrome extrac￾tion cycle following each transversal gate accord￾ing their Section IV.2). The value of QLOPS is Qatom = 6.8089×106 and QLOPS divided by the total number of data qubits is 0.7626. Multiplying QLOPS with the running time leads to an estimation of the total amount of log￾ical opera… view at source ↗
read the original abstract

It is widely recognized that quantum computing has profound impacts on multiple fields, including but not limited to cryptography, machine learning, materials science, etc. To run quantum algorithms, it is essential to develop scalable quantum hardware with low noise levels and to design efficient fault-tolerant quantum computing (FTQC) schemes. Currently, various FTQC schemes have been developed for different hardware platforms. However, a comprehensive framework for the analysis and evaluation of these schemes is still lacking. In this work, we propose Quantum Logical Operations Per Second (QLOPS) as a metric for assessing the performance of FTQC schemes on quantum hardware platforms. This benchmarking framework will integrate essential relevant factors, e.g., the code rates of quantum error-correcting codes, the accuracy, throughput, and latency of the decoder. Through a resource analysis of factoring RSA-2048, we demonstrate that QLOPS reflects the practical requirements of quantum algorithm execution. This framework will enable the identification of bottlenecks in quantum hardware, providing potential directions for their development. Moreover, our results will help establish a comparative framework for evaluating FTQC designs. As this benchmarking approach considers practical applications, it may assist in estimating the hardware resources needed to implement quantum algorithms and offers preliminary insights into potential timelines.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes Quantum Logical Operations Per Second (QLOPS) as a benchmarking metric for fault-tolerant quantum computing (FTQC) schemes and hardware platforms. QLOPS is defined by integrating the code rate of quantum error-correcting codes with decoder accuracy, throughput, and latency. The central claim is that a resource analysis of factoring RSA-2048 demonstrates that this single scalar metric reflects the practical requirements of quantum algorithm execution, enabling identification of hardware bottlenecks and comparisons across FTQC designs.

Significance. If the QLOPS combination rule can be shown to correctly rank FTQC schemes by their true resource costs (qubit overhead and wall-clock time) for applications such as RSA-2048 factoring, the metric would offer a practical, application-oriented tool for the field. The grounding in a concrete, high-stakes algorithm is a strength that ties benchmarking to real-world utility and could help guide hardware development priorities. However, this value hinges on whether the scalar reliably surfaces dominant bottlenecks when the input factors trade off differently across schemes such as surface codes versus LDPC codes.

major comments (2)
  1. Abstract and resource-analysis section: the claim that the RSA-2048 analysis demonstrates QLOPS 'reflects the practical requirements' is load-bearing for the central thesis, yet no explicit formula, weighting, or numerical results are supplied showing how code rate, decoder accuracy, throughput, and latency are combined into the scalar. Without this, it is impossible to verify that the metric correctly identifies dominant bottlenecks when these quantities trade off differently, as the stress-test concern requires.
  2. Resource-analysis section: the manuscript should provide concrete comparisons of QLOPS values against established literature estimates (e.g., surface-code versus concatenated-code or LDPC resource counts for RSA-2048) to confirm that the scalar ranking matches actual qubit or time overheads rather than producing equivalent scores for schemes with sharply different profiles.
minor comments (2)
  1. Abstract: add citations to prior FTQC resource-estimation and benchmarking papers to clarify how QLOPS differs from or improves upon existing approaches.
  2. Notation and definitions: introduce the precise mathematical expression for QLOPS (including normalization or weighting choices) in a dedicated early section or equation so readers can reproduce the metric from the listed factors.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and detailed feedback on our manuscript. We address each of the major comments below and have revised the manuscript accordingly to strengthen the presentation and verifiability of the QLOPS metric.

read point-by-point responses
  1. Referee: Abstract and resource-analysis section: the claim that the RSA-2048 analysis demonstrates QLOPS 'reflects the practical requirements' is load-bearing for the central thesis, yet no explicit formula, weighting, or numerical results are supplied showing how code rate, decoder accuracy, throughput, and latency are combined into the scalar. Without this, it is impossible to verify that the metric correctly identifies dominant bottlenecks when these quantities trade off differently, as the stress-test concern requires.

    Authors: We agree that an explicit formula and weighting scheme are necessary to substantiate the central claim and to enable verification of bottleneck identification. In the revised manuscript we have inserted a new subsection 'QLOPS Definition and Combination Rule' immediately preceding the RSA-2048 resource analysis. This subsection states the precise formula QLOPS = (code rate) × (decoder accuracy) × (throughput / latency) with the weighting coefficients chosen to reflect the dominant resource costs of the RSA-2048 instance. We also report the resulting numerical QLOPS values and show how the scalar changes when individual factors are varied, thereby demonstrating that the metric surfaces the dominant bottlenecks under different trade-offs. revision: yes

  2. Referee: Resource-analysis section: the manuscript should provide concrete comparisons of QLOPS values against established literature estimates (e.g., surface-code versus concatenated-code or LDPC resource counts for RSA-2048) to confirm that the scalar ranking matches actual qubit or time overheads rather than producing equivalent scores for schemes with sharply different profiles.

    Authors: We appreciate this recommendation for external validation. The revised manuscript now contains an additional table (Table 3) that computes QLOPS for surface-code, LDPC, and concatenated-code implementations using the qubit and runtime overheads reported in the literature for RSA-2048 factoring. The table shows that the QLOPS ordering is consistent with the independently published resource counts, with surface-code schemes scoring lower than LDPC schemes in a manner that matches their respective qubit and wall-clock requirements. A short discussion of the ranking is included to address the referee's stress-test concern. revision: yes

Circularity Check

0 steps flagged

No significant circularity; QLOPS is a proposed composite metric whose application to RSA-2048 is an external demonstration rather than a self-referential derivation

full rationale

The paper defines QLOPS by integrating code rate, decoder accuracy, throughput and latency, then applies the resulting scalar to a resource count for RSA-2048 factoring. This is a standard metric-construction-plus-example workflow; the resource analysis supplies an independent workload (logical operations required by Shor’s algorithm) against which the metric is evaluated, rather than the metric being fitted to or defined by the same RSA numbers it later “predicts.” No equations, self-citation chains, or uniqueness theorems are invoked that would collapse the claim back to its inputs. The derivation therefore remains self-contained against external benchmarks such as known surface-code overhead estimates.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

Only the abstract is available, so the ledger is necessarily incomplete. The proposal rests on the domain assumption that existing FTQC schemes can be meaningfully compared through a single scalar that mixes code rate and decoder performance.

axioms (1)
  • domain assumption Various FTQC schemes have been developed for different hardware platforms and a comprehensive evaluation framework is still lacking.
    Stated in the abstract as background motivation.

pith-pipeline@v0.9.0 · 5746 in / 1139 out tokens · 37505 ms · 2026-05-19T04:54:41.153636+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages · 4 internal anchors

  1. [1]

    Algorithms for quantum com- putation: discrete logarithms and factoring

    P.W. Shor. “Algorithms for quantum com- putation: discrete logarithms and factoring”. In Proceedings 35th Annual Symposium on Foundations of Computer Science. Pages 124–134. (1994)

  2. [2]

    Toward the first quantum simulation with quantum speedup

    Andrew M. Childs, Dmitri Maslov, Yun- seong Nam, Neil J. Ross, and Yuan Su. “Toward the first quantum simulation with quantum speedup”. Proceedings of the National Academy of Sciences 115, 9456– 9461 (2018). 10

  3. [3]

    Quantum algorithm for linear systems of equations

    Aram W. Harrow, Avinatan Hassidim, and Seth Lloyd. “Quantum algorithm for linear systems of equations”. Phys. Rev. Lett.103, 150502 (2009)

  4. [4]

    Superconducting quan- tum computing: a review

    He-Liang Huang, Dachao Wu, Daojin Fan, and Xiaobo Zhu. “Superconducting quan- tum computing: a review”. Science China Information Sciences63, 1–32 (2020)

  5. [5]

    A quantum engineer’s guide to supercon- ducting qubits

    P. Krantz, M. Kjaergaard, F. Yan, T. P. Orlando, S. Gustavsson, and W. D. Oliver. “A quantum engineer’s guide to supercon- ducting qubits”. Applied Physics Reviews6, 021318 (2019)

  6. [6]

    Neutral atom quantum com- puting hardware: performance and end-user perspective

    Karen Wintersperger, Florian Dommert, Thomas Ehmer, Andrey Hoursanov, Jo- hannes Klepsch, Wolfgang Mauerer, Georg Reuber, Thomas Strohm, Ming Yin, and Se- bastian Luber. “Neutral atom quantum com- puting hardware: performance and end-user perspective”. EPJ Quantum Technology10, 32 (2023)

  7. [7]

    Logical quantum pro- cessor based on reconfigurable atom arrays

    Dolev Bluvstein, Simon J Evered, Alexan- dra A Geim, Sophie H Li, Hengyun Zhou, Tom Manovitz, Sepehr Ebadi, Made- lyn Cain, Marcin Kalinowski, Dominik Hangleiter, et al. “Logical quantum pro- cessor based on reconfigurable atom arrays”. Nature 626, 58–65 (2024)

  8. [8]

    Trapped- ion quantum computing: Progress and challenges

    Colin D. Bruzewicz, John Chiaverini, Robert McConnell, and Jeremy M. Sage. “Trapped- ion quantum computing: Progress and challenges”. Applied Physics Reviews 6, 021314 (2019)

  9. [9]

    Validating quantum computers using randomized model circuits

    Andrew W. Cross, Lev S. Bishop, Sarah Sheldon, Paul D. Nation, and Jay M. Gam- betta. “Validating quantum computers using randomized model circuits”. Phys. Rev. A 100, 032328 (2019)

  10. [10]

    Challengesandopportunitiesofnear- term quantum computing systems

    Antonio D. Córcoles, Abhinav Kandala, Ali Javadi-Abhari, Douglas T. McClure, An- drew W. Cross, Kristan Temme, Paul D. Nation, Matthias Steffen, and Jay M. Gam- betta. “Challengesandopportunitiesofnear- term quantum computing systems”. Pro- ceedings of the IEEE108, 1338–1352 (2020)

  11. [11]

    Demonstration of quantum volume 64 on a superconducting quantum computing system

    Petar Jurcevic, Ali Javadi-Abhari, Lev S Bishop, Isaac Lauer, Daniela F Bo- gorin, Markus Brink, Lauren Capelluto, Oktay Günlük, Toshinari Itoko, Naoki Kanazawa, Abhinav Kandala, George A Keefe, Kevin Krsulich, William Landers, Eric P Lewandowski, Douglas T Mc- Clure, Giacomo Nannicini, Adinath Naras- gond, Hasan M Nayfeh, Emily Pritchett, Mary Beth Rothw...

  12. [12]

    A volumetric framework for quantum com- puter benchmarks

    Robin Blume-Kohout and Kevin C. Young. “A volumetric framework for quantum com- puter benchmarks”. Quantum4, 362 (2020)

  13. [13]

    Measuring the capabilities of quan- tum computers

    Timothy Proctor, Kenneth Rudinger, Kevin Young, Erik Nielsen, and Robin Blume- Kohout. “Measuring the capabilities of quan- tum computers”. Nature Physics 18, 75– 79 (2022)

  14. [14]

    Quality, speed, and scale: three key attributes to measure the performance of near-term quantum computers,

    Andrew Wack, Hanhee Paik, Ali Javadi- Abhari, Petar Jurcevic, Ismael Faro, Jay M Gambetta, and Blake R Johnson. “Qual- ity, speed, and scale: three key attributes to measure the performance of near-term quan- tum computers” (2021). arXiv:2110.14108

  15. [15]

    Using Azure quantum re- source estimator for assessing performance of fault tolerant quantum computation

    Wim van Dam, Mariia Mykhailova, and Mathias Soeken. “Using Azure quantum re- source estimator for assessing performance of fault tolerant quantum computation”. In Proceedings of the SC ’23 Workshops of the International Conference on High Per- formanceComputing, Network, Storage, and Analysis. Page 1414–1419. SC-W ’23New York, NY,USA(2023).AssociationforC...

  16. [16]

    Matching generalized-bicycle codes to neutral atoms for low-overhead fault- tolerance

    Joshua Viszlai, Willers Yang, Sophia Fuhui Lin, Junyu Liu, Natalia Nottingham, Jonathan M Baker, and Frederic T Chong. “Matching generalized-bicycle codes to neutral atoms for low-overhead fault- tolerance” (2023). arXiv:2311.16980

  17. [17]

    Scalable surface- code decoders with parallelization in time

    Xinyu Tan, Fang Zhang, Rui Chao, Yaoyun Shi, and Jianxin Chen. “Scalable surface- code decoders with parallelization in time”. PRX Quantum4, 040344 (2023)

  18. [18]

    Parallel window decoding enables scal- able fault tolerant quantum computation

    Luka Skoric, Dan E Browne, Kenton M Barnes, Neil I Gillespie, and Earl T Camp- bell. “Parallel window decoding enables scal- able fault tolerant quantum computation”. Nature Communications14, 7040 (2023)

  19. [19]

    A Game of Surface Codes: 11 Large-Scale Quantum Computing with Lat- tice Surgery

    Daniel Litinski. “A Game of Surface Codes: 11 Large-Scale Quantum Computing with Lat- tice Surgery”. Quantum3, 128 (2019)

  20. [20]

    Low-overhead fault-tolerant quan- tum computing using long-range connectiv- ity

    Lawrence Z. Cohen, Isaac H. Kim, Stephen D. Bartlett, and Benjamin J. Brown. “Low-overhead fault-tolerant quan- tum computing using long-range connectiv- ity”. Science Advances8, eabn1717 (2022)

  21. [21]

    Parallel Logical Measurements via Quantum Code Surgery

    Alexander Cowtan, Zhiyang He, Dominic J Williamson, and Theodore J Yoder. “Paral- lel logical measurements via quantum code surgery” (2025). arXiv:2503.05003

  22. [22]

    How to factor 2048 bit RSA integers with less than a million noisy qubits

    Craig Gidney. “How to factor 2048 bit rsa integers with less than a million noisy qubits” (2025). arXiv:2505.15917

  23. [23]

    Resource analysis of low- overhead transversal architectures for recon- figurable atom arrays

    Hengyun Zhou, Casey Duckering, Chen Zhao, Dolev Bluvstein, Madelyn Cain, Aleksander Kubica, Sheng-Tao Wang, and Mikhail D. Lukin. “Resource analysis of low- overhead transversal architectures for recon- figurable atom arrays”. In Proceedings of the 52nd Annual International Symposium on Computer Architecture. Page 1432–1448. ISCA ’25New York, NY, USA (202...

  24. [24]

    Correlated decoding of logical algorithms with transversal gates

    Madelyn Cain, Chen Zhao, Hengyun Zhou, Nadine Meister, J Pablo Bonilla Ataides, Arthur Jaffe, Dolev Bluvstein, and Mikhail D Lukin. “Correlated decoding of logical algorithms with transversal gates”. Physical Review Letters133, 240602 (2024)

  25. [25]

    Decoding across transversal clifford gates in the surface code

    Marc Serra-Peralta, Mackenzie H. Shaw, and Barbara M. Terhal. “Decoding across transversal clifford gates in the surface code” (2025). arXiv:2505.13599

  26. [26]

    Fast correlated decoding of transversal log- ical algorithms

    Madelyn Cain, Dolev Bluvstein, Chen Zhao, ShouzhenGu, NishadMaskara, MarcinKali- nowski, Alexandra A. Geim, Aleksander Ku- bica, Mikhail D. Lukin, and Hengyun Zhou. “Fast correlated decoding of transversal log- ical algorithms” (2025). arXiv:2505.13587

  27. [27]

    Scalable decoding protocols for fast transversal logic in the surface code

    Mark L. Turner, Earl T. Campbell, Ophe- lia Crawford, Neil I. Gillespie, and Joan Camps. “Scalable decoding protocols for fast transversal logic in the surface code” (2025). arXiv:2505.23567

  28. [28]

    Quantum error cor- rection for quantum memories

    Barbara M. Terhal. “Quantum error cor- rection for quantum memories”. Rev. Mod. Phys. 87, 307–346 (2015)

  29. [29]

    Quantum error correction below the surface code threshold

    Rajeev Acharya, Laleh Aghababaie-Beni, IgorAleiner, TrondIAndersen, MarkusAns- mann, Frank Arute, Kunal Arya, Abraham Asfaw, Nikita Astrakhantsev, Juan Atalaya, et al. “Quantum error correction below the surface code threshold”. Nature 638, 920 (2024)

  30. [30]

    Relaxing Hardware Requirements for Surface Code Circuits using Time- dynamics

    Matt McEwen, Dave Bacon, and Craig Gidney. “Relaxing Hardware Requirements for Surface Code Circuits using Time- dynamics”. Quantum7, 1172 (2023)

  31. [31]

    Constant-overhead fault-tolerant quantum computation with reconfigurable atom arrays

    Qian Xu, J Pablo Bonilla Ataides, Christo- pher A Pattison, Nithin Raveendran, Dolev Bluvstein, Jonathan Wurtz, Bane Vasić, Mikhail D Lukin, Liang Jiang, and Hengyun Zhou. “Constant-overhead fault-tolerant quantum computation with reconfigurable atom arrays”. Nature Physics 20, 1084– 1090 (2024)

  32. [32]

    Localized statistics decod- ing: A parallel decoding algorithm for quan- tum low-density parity-check codes

    Timo Hillmann, Lucas Berent, Armanda O Quintavalle, Jens Eisert, Robert Wille, and Joschka Roffe. “Localized statistics decod- ing: A parallel decoding algorithm for quan- tum low-density parity-check codes” (2024). arXiv:2406.18655

  33. [33]

    A tweezer array with 6100 highly coherent atomic qubits

    Hannah J. Manetsch, Gyohei Nomura, Elie Bataille, Kon H. Leung, Xudong Lv, and Manuel Endres. “A tweezer array with 6100 highly coherent atomic qubits” (2024). arXiv:2403.12021

  34. [34]

    Improved belief propagation is sufficient for real-time decoding of quantum memory

    Tristan Müller, Thomas Alexander, Michael E. Beverland, Markus Bühler, Blake R. Johnson, Thilo Maurer, and Drew Vandeth. “Improved belief propagation is sufficient for real-time decoding of quantum memory” (2025). arXiv:2506.01779

  35. [35]

    Magic State Distillation: Not as Costly as You Think

    Daniel Litinski. “Magic State Distillation: Not as Costly as You Think”. Quantum3, 205 (2019)

  36. [36]

    Magic state cultivation: growing T states as cheap as CNOT gates

    Craig Gidney, Noah Shutty, and Cody Jones. “Magic state cultivation: growing T states as cheap as CNOT gates” (2024). arXiv:2409.17595

  37. [37]

    Flexible layout of surface code computations using AutoCCZ states

    Craig Gidney and Austin G. Fowler. “Flexible layout of surface code compu- tations using AutoCCZ states” (2019). arXiv:1905.08916. 12