Benchmarking fault-tolerant quantum computing hardware via QLOPS
Pith reviewed 2026-05-19 04:54 UTC · model grok-4.3
The pith
QLOPS metric benchmarks fault-tolerant quantum hardware by factoring RSA-2048 resource needs.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
QLOPS is defined as a scalar that multiplies the logical operation rate by the effective code rate while dividing by factors that capture decoder error rate, throughput, and latency. Resource analysis of Shor's algorithm for factoring RSA-2048 shows that this quantity directly measures the time and hardware overhead required to obtain the correct answer, thereby revealing which hardware features dominate the cost of fault-tolerant execution.
What carries the argument
QLOPS, the scalar formed by combining quantum error-correcting code rate with decoder accuracy, throughput, and latency to quantify the rate of useful logical operations.
If this is right
- Hardware teams can rank proposed improvements by their expected effect on QLOPS rather than by isolated metrics such as physical error rate alone.
- Designers obtain concrete targets for decoder speed and accuracy needed to reach a given logical operation rate.
- Different error-correcting codes can be compared directly on the basis of the QLOPS they deliver for the same algorithm.
- Rough timelines for when factoring or other algorithms become feasible follow from projecting how QLOPS scales with hardware progress.
- Resource estimates for large algorithms become more realistic because they now include decoder overhead explicitly.
Where Pith is reading between the lines
- The same QLOPS framework could be applied to other algorithms such as quantum simulation or optimization to check whether the bottlenecks identified for factoring are algorithm-specific.
- Hardware roadmaps might shift emphasis toward faster classical control systems once QLOPS shows that decoder latency often limits overall performance.
- Public benchmarks using QLOPS could let experimental groups publish comparable figures even when they use different error-correcting codes.
Load-bearing premise
Combining code rate, decoder accuracy, throughput, and latency into one scalar will reliably identify the dominant bottlenecks across different fault-tolerant schemes and hardware platforms.
What would settle it
Two FTQC schemes with different QLOPS values are run on the same hardware; if the lower-QLOPS scheme finishes RSA-2048 factoring faster or with fewer physical resources, the metric does not reflect practical requirements.
Figures
read the original abstract
It is widely recognized that quantum computing has profound impacts on multiple fields, including but not limited to cryptography, machine learning, materials science, etc. To run quantum algorithms, it is essential to develop scalable quantum hardware with low noise levels and to design efficient fault-tolerant quantum computing (FTQC) schemes. Currently, various FTQC schemes have been developed for different hardware platforms. However, a comprehensive framework for the analysis and evaluation of these schemes is still lacking. In this work, we propose Quantum Logical Operations Per Second (QLOPS) as a metric for assessing the performance of FTQC schemes on quantum hardware platforms. This benchmarking framework will integrate essential relevant factors, e.g., the code rates of quantum error-correcting codes, the accuracy, throughput, and latency of the decoder. Through a resource analysis of factoring RSA-2048, we demonstrate that QLOPS reflects the practical requirements of quantum algorithm execution. This framework will enable the identification of bottlenecks in quantum hardware, providing potential directions for their development. Moreover, our results will help establish a comparative framework for evaluating FTQC designs. As this benchmarking approach considers practical applications, it may assist in estimating the hardware resources needed to implement quantum algorithms and offers preliminary insights into potential timelines.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes Quantum Logical Operations Per Second (QLOPS) as a benchmarking metric for fault-tolerant quantum computing (FTQC) schemes and hardware platforms. QLOPS is defined by integrating the code rate of quantum error-correcting codes with decoder accuracy, throughput, and latency. The central claim is that a resource analysis of factoring RSA-2048 demonstrates that this single scalar metric reflects the practical requirements of quantum algorithm execution, enabling identification of hardware bottlenecks and comparisons across FTQC designs.
Significance. If the QLOPS combination rule can be shown to correctly rank FTQC schemes by their true resource costs (qubit overhead and wall-clock time) for applications such as RSA-2048 factoring, the metric would offer a practical, application-oriented tool for the field. The grounding in a concrete, high-stakes algorithm is a strength that ties benchmarking to real-world utility and could help guide hardware development priorities. However, this value hinges on whether the scalar reliably surfaces dominant bottlenecks when the input factors trade off differently across schemes such as surface codes versus LDPC codes.
major comments (2)
- Abstract and resource-analysis section: the claim that the RSA-2048 analysis demonstrates QLOPS 'reflects the practical requirements' is load-bearing for the central thesis, yet no explicit formula, weighting, or numerical results are supplied showing how code rate, decoder accuracy, throughput, and latency are combined into the scalar. Without this, it is impossible to verify that the metric correctly identifies dominant bottlenecks when these quantities trade off differently, as the stress-test concern requires.
- Resource-analysis section: the manuscript should provide concrete comparisons of QLOPS values against established literature estimates (e.g., surface-code versus concatenated-code or LDPC resource counts for RSA-2048) to confirm that the scalar ranking matches actual qubit or time overheads rather than producing equivalent scores for schemes with sharply different profiles.
minor comments (2)
- Abstract: add citations to prior FTQC resource-estimation and benchmarking papers to clarify how QLOPS differs from or improves upon existing approaches.
- Notation and definitions: introduce the precise mathematical expression for QLOPS (including normalization or weighting choices) in a dedicated early section or equation so readers can reproduce the metric from the listed factors.
Simulated Author's Rebuttal
We thank the referee for their constructive and detailed feedback on our manuscript. We address each of the major comments below and have revised the manuscript accordingly to strengthen the presentation and verifiability of the QLOPS metric.
read point-by-point responses
-
Referee: Abstract and resource-analysis section: the claim that the RSA-2048 analysis demonstrates QLOPS 'reflects the practical requirements' is load-bearing for the central thesis, yet no explicit formula, weighting, or numerical results are supplied showing how code rate, decoder accuracy, throughput, and latency are combined into the scalar. Without this, it is impossible to verify that the metric correctly identifies dominant bottlenecks when these quantities trade off differently, as the stress-test concern requires.
Authors: We agree that an explicit formula and weighting scheme are necessary to substantiate the central claim and to enable verification of bottleneck identification. In the revised manuscript we have inserted a new subsection 'QLOPS Definition and Combination Rule' immediately preceding the RSA-2048 resource analysis. This subsection states the precise formula QLOPS = (code rate) × (decoder accuracy) × (throughput / latency) with the weighting coefficients chosen to reflect the dominant resource costs of the RSA-2048 instance. We also report the resulting numerical QLOPS values and show how the scalar changes when individual factors are varied, thereby demonstrating that the metric surfaces the dominant bottlenecks under different trade-offs. revision: yes
-
Referee: Resource-analysis section: the manuscript should provide concrete comparisons of QLOPS values against established literature estimates (e.g., surface-code versus concatenated-code or LDPC resource counts for RSA-2048) to confirm that the scalar ranking matches actual qubit or time overheads rather than producing equivalent scores for schemes with sharply different profiles.
Authors: We appreciate this recommendation for external validation. The revised manuscript now contains an additional table (Table 3) that computes QLOPS for surface-code, LDPC, and concatenated-code implementations using the qubit and runtime overheads reported in the literature for RSA-2048 factoring. The table shows that the QLOPS ordering is consistent with the independently published resource counts, with surface-code schemes scoring lower than LDPC schemes in a manner that matches their respective qubit and wall-clock requirements. A short discussion of the ranking is included to address the referee's stress-test concern. revision: yes
Circularity Check
No significant circularity; QLOPS is a proposed composite metric whose application to RSA-2048 is an external demonstration rather than a self-referential derivation
full rationale
The paper defines QLOPS by integrating code rate, decoder accuracy, throughput and latency, then applies the resulting scalar to a resource count for RSA-2048 factoring. This is a standard metric-construction-plus-example workflow; the resource analysis supplies an independent workload (logical operations required by Shor’s algorithm) against which the metric is evaluated, rather than the metric being fitted to or defined by the same RSA numbers it later “predicts.” No equations, self-citation chains, or uniqueness theorems are invoked that would collapse the claim back to its inputs. The derivation therefore remains self-contained against external benchmarks such as known surface-code overhead estimates.
Axiom & Free-Parameter Ledger
axioms (1)
- domain assumption Various FTQC schemes have been developed for different hardware platforms and a comprehensive evaluation framework is still lacking.
Lean theorems connected to this paper
-
IndisputableMonolith/Cost/FunctionalEquation.leanwashburn_uniqueness_aczel unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
We propose Quantum Logical Operations Per Second (QLOPS) as a metric... integrates... code rates... accuracy, throughput, and latency of the decoder... resource analysis of factoring RSA-2048
-
IndisputableMonolith/Foundation/ArithmeticFromLogic.leanLogicNat unclear?
unclearRelation between the paper passage and the cited Recognition theorem.
Q = k× 1/(⌈tr/tSEC⌉+ d)tSEC
What do these tags mean?
- matches
- The paper's claim is directly supported by a theorem in the formal canon.
- supports
- The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
- extends
- The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
- uses
- The paper appears to rely on the theorem as machinery.
- contradicts
- The paper's claim conflicts with a theorem or certificate in the canon.
- unclear
- Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.
Reference graph
Works this paper leans on
-
[1]
Algorithms for quantum com- putation: discrete logarithms and factoring
P.W. Shor. “Algorithms for quantum com- putation: discrete logarithms and factoring”. In Proceedings 35th Annual Symposium on Foundations of Computer Science. Pages 124–134. (1994)
work page 1994
-
[2]
Toward the first quantum simulation with quantum speedup
Andrew M. Childs, Dmitri Maslov, Yun- seong Nam, Neil J. Ross, and Yuan Su. “Toward the first quantum simulation with quantum speedup”. Proceedings of the National Academy of Sciences 115, 9456– 9461 (2018). 10
work page 2018
-
[3]
Quantum algorithm for linear systems of equations
Aram W. Harrow, Avinatan Hassidim, and Seth Lloyd. “Quantum algorithm for linear systems of equations”. Phys. Rev. Lett.103, 150502 (2009)
work page 2009
-
[4]
Superconducting quan- tum computing: a review
He-Liang Huang, Dachao Wu, Daojin Fan, and Xiaobo Zhu. “Superconducting quan- tum computing: a review”. Science China Information Sciences63, 1–32 (2020)
work page 2020
-
[5]
A quantum engineer’s guide to supercon- ducting qubits
P. Krantz, M. Kjaergaard, F. Yan, T. P. Orlando, S. Gustavsson, and W. D. Oliver. “A quantum engineer’s guide to supercon- ducting qubits”. Applied Physics Reviews6, 021318 (2019)
work page 2019
-
[6]
Neutral atom quantum com- puting hardware: performance and end-user perspective
Karen Wintersperger, Florian Dommert, Thomas Ehmer, Andrey Hoursanov, Jo- hannes Klepsch, Wolfgang Mauerer, Georg Reuber, Thomas Strohm, Ming Yin, and Se- bastian Luber. “Neutral atom quantum com- puting hardware: performance and end-user perspective”. EPJ Quantum Technology10, 32 (2023)
work page 2023
-
[7]
Logical quantum pro- cessor based on reconfigurable atom arrays
Dolev Bluvstein, Simon J Evered, Alexan- dra A Geim, Sophie H Li, Hengyun Zhou, Tom Manovitz, Sepehr Ebadi, Made- lyn Cain, Marcin Kalinowski, Dominik Hangleiter, et al. “Logical quantum pro- cessor based on reconfigurable atom arrays”. Nature 626, 58–65 (2024)
work page 2024
-
[8]
Trapped- ion quantum computing: Progress and challenges
Colin D. Bruzewicz, John Chiaverini, Robert McConnell, and Jeremy M. Sage. “Trapped- ion quantum computing: Progress and challenges”. Applied Physics Reviews 6, 021314 (2019)
work page 2019
-
[9]
Validating quantum computers using randomized model circuits
Andrew W. Cross, Lev S. Bishop, Sarah Sheldon, Paul D. Nation, and Jay M. Gam- betta. “Validating quantum computers using randomized model circuits”. Phys. Rev. A 100, 032328 (2019)
work page 2019
-
[10]
Challengesandopportunitiesofnear- term quantum computing systems
Antonio D. Córcoles, Abhinav Kandala, Ali Javadi-Abhari, Douglas T. McClure, An- drew W. Cross, Kristan Temme, Paul D. Nation, Matthias Steffen, and Jay M. Gam- betta. “Challengesandopportunitiesofnear- term quantum computing systems”. Pro- ceedings of the IEEE108, 1338–1352 (2020)
work page 2020
-
[11]
Demonstration of quantum volume 64 on a superconducting quantum computing system
Petar Jurcevic, Ali Javadi-Abhari, Lev S Bishop, Isaac Lauer, Daniela F Bo- gorin, Markus Brink, Lauren Capelluto, Oktay Günlük, Toshinari Itoko, Naoki Kanazawa, Abhinav Kandala, George A Keefe, Kevin Krsulich, William Landers, Eric P Lewandowski, Douglas T Mc- Clure, Giacomo Nannicini, Adinath Naras- gond, Hasan M Nayfeh, Emily Pritchett, Mary Beth Rothw...
work page 2021
-
[12]
A volumetric framework for quantum com- puter benchmarks
Robin Blume-Kohout and Kevin C. Young. “A volumetric framework for quantum com- puter benchmarks”. Quantum4, 362 (2020)
work page 2020
-
[13]
Measuring the capabilities of quan- tum computers
Timothy Proctor, Kenneth Rudinger, Kevin Young, Erik Nielsen, and Robin Blume- Kohout. “Measuring the capabilities of quan- tum computers”. Nature Physics 18, 75– 79 (2022)
work page 2022
-
[14]
Andrew Wack, Hanhee Paik, Ali Javadi- Abhari, Petar Jurcevic, Ismael Faro, Jay M Gambetta, and Blake R Johnson. “Qual- ity, speed, and scale: three key attributes to measure the performance of near-term quan- tum computers” (2021). arXiv:2110.14108
-
[15]
Wim van Dam, Mariia Mykhailova, and Mathias Soeken. “Using Azure quantum re- source estimator for assessing performance of fault tolerant quantum computation”. In Proceedings of the SC ’23 Workshops of the International Conference on High Per- formanceComputing, Network, Storage, and Analysis. Page 1414–1419. SC-W ’23New York, NY,USA(2023).AssociationforC...
work page 2023
-
[16]
Matching generalized-bicycle codes to neutral atoms for low-overhead fault- tolerance
Joshua Viszlai, Willers Yang, Sophia Fuhui Lin, Junyu Liu, Natalia Nottingham, Jonathan M Baker, and Frederic T Chong. “Matching generalized-bicycle codes to neutral atoms for low-overhead fault- tolerance” (2023). arXiv:2311.16980
-
[17]
Scalable surface- code decoders with parallelization in time
Xinyu Tan, Fang Zhang, Rui Chao, Yaoyun Shi, and Jianxin Chen. “Scalable surface- code decoders with parallelization in time”. PRX Quantum4, 040344 (2023)
work page 2023
-
[18]
Parallel window decoding enables scal- able fault tolerant quantum computation
Luka Skoric, Dan E Browne, Kenton M Barnes, Neil I Gillespie, and Earl T Camp- bell. “Parallel window decoding enables scal- able fault tolerant quantum computation”. Nature Communications14, 7040 (2023)
work page 2023
-
[19]
A Game of Surface Codes: 11 Large-Scale Quantum Computing with Lat- tice Surgery
Daniel Litinski. “A Game of Surface Codes: 11 Large-Scale Quantum Computing with Lat- tice Surgery”. Quantum3, 128 (2019)
work page 2019
-
[20]
Low-overhead fault-tolerant quan- tum computing using long-range connectiv- ity
Lawrence Z. Cohen, Isaac H. Kim, Stephen D. Bartlett, and Benjamin J. Brown. “Low-overhead fault-tolerant quan- tum computing using long-range connectiv- ity”. Science Advances8, eabn1717 (2022)
work page 2022
-
[21]
Parallel Logical Measurements via Quantum Code Surgery
Alexander Cowtan, Zhiyang He, Dominic J Williamson, and Theodore J Yoder. “Paral- lel logical measurements via quantum code surgery” (2025). arXiv:2503.05003
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[22]
How to factor 2048 bit RSA integers with less than a million noisy qubits
Craig Gidney. “How to factor 2048 bit rsa integers with less than a million noisy qubits” (2025). arXiv:2505.15917
work page internal anchor Pith review Pith/arXiv arXiv 2048
-
[23]
Resource analysis of low- overhead transversal architectures for recon- figurable atom arrays
Hengyun Zhou, Casey Duckering, Chen Zhao, Dolev Bluvstein, Madelyn Cain, Aleksander Kubica, Sheng-Tao Wang, and Mikhail D. Lukin. “Resource analysis of low- overhead transversal architectures for recon- figurable atom arrays”. In Proceedings of the 52nd Annual International Symposium on Computer Architecture. Page 1432–1448. ISCA ’25New York, NY, USA (202...
work page 2025
-
[24]
Correlated decoding of logical algorithms with transversal gates
Madelyn Cain, Chen Zhao, Hengyun Zhou, Nadine Meister, J Pablo Bonilla Ataides, Arthur Jaffe, Dolev Bluvstein, and Mikhail D Lukin. “Correlated decoding of logical algorithms with transversal gates”. Physical Review Letters133, 240602 (2024)
work page 2024
-
[25]
Decoding across transversal clifford gates in the surface code
Marc Serra-Peralta, Mackenzie H. Shaw, and Barbara M. Terhal. “Decoding across transversal clifford gates in the surface code” (2025). arXiv:2505.13599
-
[26]
Fast correlated decoding of transversal log- ical algorithms
Madelyn Cain, Dolev Bluvstein, Chen Zhao, ShouzhenGu, NishadMaskara, MarcinKali- nowski, Alexandra A. Geim, Aleksander Ku- bica, Mikhail D. Lukin, and Hengyun Zhou. “Fast correlated decoding of transversal log- ical algorithms” (2025). arXiv:2505.13587
-
[27]
Scalable decoding protocols for fast transversal logic in the surface code
Mark L. Turner, Earl T. Campbell, Ophe- lia Crawford, Neil I. Gillespie, and Joan Camps. “Scalable decoding protocols for fast transversal logic in the surface code” (2025). arXiv:2505.23567
-
[28]
Quantum error cor- rection for quantum memories
Barbara M. Terhal. “Quantum error cor- rection for quantum memories”. Rev. Mod. Phys. 87, 307–346 (2015)
work page 2015
-
[29]
Quantum error correction below the surface code threshold
Rajeev Acharya, Laleh Aghababaie-Beni, IgorAleiner, TrondIAndersen, MarkusAns- mann, Frank Arute, Kunal Arya, Abraham Asfaw, Nikita Astrakhantsev, Juan Atalaya, et al. “Quantum error correction below the surface code threshold”. Nature 638, 920 (2024)
work page 2024
-
[30]
Relaxing Hardware Requirements for Surface Code Circuits using Time- dynamics
Matt McEwen, Dave Bacon, and Craig Gidney. “Relaxing Hardware Requirements for Surface Code Circuits using Time- dynamics”. Quantum7, 1172 (2023)
work page 2023
-
[31]
Constant-overhead fault-tolerant quantum computation with reconfigurable atom arrays
Qian Xu, J Pablo Bonilla Ataides, Christo- pher A Pattison, Nithin Raveendran, Dolev Bluvstein, Jonathan Wurtz, Bane Vasić, Mikhail D Lukin, Liang Jiang, and Hengyun Zhou. “Constant-overhead fault-tolerant quantum computation with reconfigurable atom arrays”. Nature Physics 20, 1084– 1090 (2024)
work page 2024
-
[32]
Timo Hillmann, Lucas Berent, Armanda O Quintavalle, Jens Eisert, Robert Wille, and Joschka Roffe. “Localized statistics decod- ing: A parallel decoding algorithm for quan- tum low-density parity-check codes” (2024). arXiv:2406.18655
-
[33]
A tweezer array with 6100 highly coherent atomic qubits
Hannah J. Manetsch, Gyohei Nomura, Elie Bataille, Kon H. Leung, Xudong Lv, and Manuel Endres. “A tweezer array with 6100 highly coherent atomic qubits” (2024). arXiv:2403.12021
-
[34]
Improved belief propagation is sufficient for real-time decoding of quantum memory
Tristan Müller, Thomas Alexander, Michael E. Beverland, Markus Bühler, Blake R. Johnson, Thilo Maurer, and Drew Vandeth. “Improved belief propagation is sufficient for real-time decoding of quantum memory” (2025). arXiv:2506.01779
-
[35]
Magic State Distillation: Not as Costly as You Think
Daniel Litinski. “Magic State Distillation: Not as Costly as You Think”. Quantum3, 205 (2019)
work page 2019
-
[36]
Magic state cultivation: growing T states as cheap as CNOT gates
Craig Gidney, Noah Shutty, and Cody Jones. “Magic state cultivation: growing T states as cheap as CNOT gates” (2024). arXiv:2409.17595
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[37]
Flexible layout of surface code computations using AutoCCZ states
Craig Gidney and Austin G. Fowler. “Flexible layout of surface code compu- tations using AutoCCZ states” (2019). arXiv:1905.08916. 12
work page internal anchor Pith review Pith/arXiv arXiv 2019
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.