Benchmarking quantum devices beyond classical capabilities
Pith reviewed 2026-05-23 03:32 UTC · model grok-4.3
The pith
Modified Quantum Volume test uses restricted universal circuits to identify heavy outputs directly without classical simulation.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The modified Quantum Volume test adopts a carefully restricted circuit ensemble generated from a gate set that remains universal for quantum computation, allowing direct determination of the heavy-output subspace. This overcomes the scalability barrier of the Quantum Volume test beyond classical computational limits while still probing the key features of universal quantum computing.
What carries the argument
The restricted circuit ensemble generated from a universal gate set, which enables direct identification of the heavy-output subspace.
If this is right
- Quantum Volume benchmarking becomes feasible for devices whose circuits exceed classical simulation capacity.
- The test remains architecture-independent and applicable across different quantum hardware platforms.
- The benchmark continues to assess capability for general quantum computation rather than a narrow task.
- The exponential classical cost that previously limited Quantum Volume is eliminated for the modified ensemble.
Where Pith is reading between the lines
- The same restriction technique might be applied to other benchmarks that rely on identifying probable outcomes to extend their reach beyond classical simulation.
- If the approach works, it opens the possibility of designing additional classically tractable yet universal circuit families for testing specific quantum resources.
- One could test whether performance on the modified test predicts success on concrete algorithms such as variational quantum eigensolvers on the same device.
Load-bearing premise
The restricted ensemble still captures the essential features of universal quantum computation that the original Quantum Volume test was designed to measure.
What would settle it
Running both the original and modified tests on devices with few enough qubits for full classical simulation and finding that the two tests give uncorrelated pass/fail results on the same hardware would falsify the claim that the modified version continues to probe the same key features.
Figures
read the original abstract
Rapid development of quantum computing technology has led to a wide variety of sophisticated quantum devices. Benchmarking these systems becomes crucial for understanding their capabilities and paving the way for future advancements. The Quantum Volume (QV) test is one of the most widely used benchmarks for evaluating quantum computer performance due to its architecture independence. However, as the number of qubits in a quantum device grows, the test faces a significant limitation. It requires determining the subspace of the most probable outcomes, a task that is typically performed via classical simulation of the quantum circuit and therefore incurs an exponential computational cost. In this work, we propose modifications to the QV test, by adopting a carefully restricted circuit ensemble generated from a gate set that remains universal for quantum computation, that allows for the direct determination of the heavy-output subspace. Crucially, the modified circuits remain capable of general quantum computation. This approach overcomes the scalability barrier of the Quantum Volume test beyond classical computational limits, while still probing the key features of universal quantum computing.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes modifications to the Quantum Volume (QV) test to overcome its scalability limitation. The original test requires classical simulation to identify the heavy-output subspace, incurring exponential cost as qubit number grows. The authors suggest using a carefully restricted circuit ensemble generated from a gate set that remains universal for quantum computation; this is claimed to permit direct determination of the heavy-output subspace. The modified circuits are asserted to remain capable of general quantum computation and to continue probing the key features of universal quantum computing.
Significance. If the central claim holds, the work would be significant because it would enable QV-style benchmarking of devices with qubit counts beyond the reach of classical simulation while retaining architecture independence and diagnostic power for universal QC. The absence of any explicit construction, however, prevents confirmation that the restriction achieves this without circularity or loss of generality.
major comments (2)
- [Abstract] Abstract: the claim that the restricted ensemble 'still probes the key features of universal quantum computing' is presented without any derivation, explicit gate-set definition, or verification that the restriction preserves the original test's diagnostic power; this is the load-bearing assumption for the central claim.
- [Abstract] Abstract: no circuit construction, ensemble definition, or argument is supplied showing how direct heavy-output identification is achieved while the gate set remains universal and the circuits retain general computational capability.
minor comments (1)
- The abstract is concise but contains no equations, tables, or figures, which is atypical for a technical proposal of this type and makes independent assessment impossible from the given material.
Simulated Author's Rebuttal
We thank the referee for their careful review and for identifying points where the presentation of our modified Quantum Volume test can be strengthened. We address the major comments below and will revise the manuscript accordingly.
read point-by-point responses
-
Referee: [Abstract] Abstract: the claim that the restricted ensemble 'still probes the key features of universal quantum computing' is presented without any derivation, explicit gate-set definition, or verification that the restriction preserves the original test's diagnostic power; this is the load-bearing assumption for the central claim.
Authors: We agree that the abstract, owing to length constraints, states the claim without accompanying derivation or definitions. The manuscript introduces the restricted ensemble at a conceptual level but does not supply the requested explicit gate-set definition or verification in the provided sections. We will revise by adding both an explicit gate-set definition and a derivation (including verification that diagnostic power is retained) to the main text and a concise reference in the abstract. revision: yes
-
Referee: [Abstract] Abstract: no circuit construction, ensemble definition, or argument is supplied showing how direct heavy-output identification is achieved while the gate set remains universal and the circuits retain general computational capability.
Authors: We acknowledge the absence of an explicit circuit construction, ensemble definition, or detailed argument in the abstract (and, per the referee's observation, in the manuscript as reviewed). The current text asserts the existence of such a construction but does not exhibit it. In the revision we will insert a dedicated subsection containing the circuit construction, ensemble definition, and the argument demonstrating direct heavy-output identification while preserving universality and general computational capability. revision: yes
Circularity Check
No significant circularity
full rationale
The provided abstract and description contain no equations, fitted parameters, self-citations, or derivation steps. The proposal modifies the QV test via a restricted universal gate-set ensemble to enable direct heavy-output identification. This is a methodological suggestion whose validity rests on external verification of the ensemble's properties rather than any internal reduction to inputs by construction. No load-bearing claim reduces to self-definition, renaming, or fitted prediction.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
(2−ϵ)N −1 2N −1 ( (4−3ϵ)N −1 4N −1 ) M−1 , ̃hm U = 1+(1−ϵ) mM 2 (10) (see derivation in Appendix D 3 d). Thus the information obtained from multiple ̃hm U is in fact redundant, and any of them can be used to estimate the value of ̃hU. VI. BENCHMARKING FUTURE QUANTUM COMPUTERS Natural idea for characterizing quantum computers is to use some low level bench...
-
[2]
stabilizer states [47] or tensor networks [48]. Yet those approaches are doomed to failure due to generic- ity of QV circuit, which translates to large number of non-matchgates, e.g. swaps, non-Clifford gates and large bond dimension, respectively. More insightful method to simulate the QV circuit might try to leverage this universality. An approach based...
work page 2021
- [3]
-
[4]
P. Jurcevicet al., Demonstration of quantum volume 64 on a superconducting quantum computing system, Quan- tum Sci. Technol.6, 025020 (2021)
work page 2021
-
[5]
J. M. Pino, J. M. Dreiling, C. Figgatt,et al., Demonstra- tion of the trapped-ion quantum ccd computer architec- ture, Nature592, 209–213 (2021)
work page 2021
-
[6]
Honeywell Sets Another Record For Quantum Comput- ing Performance, internet announcement (2024)
work page 2024
-
[7]
D. Bluvstein, S. J. Evered, A. A. Geim,et al., Logical quantum processor based on reconfigurable atom arrays, Nature626, 58–65 (2023)
work page 2023
-
[8]
Y. Kim, A. Eddins, S. Anand,et al., Evidence for the utility of quantum computing before fault tolerance, Na- ture618, 500 (2023)
work page 2023
-
[9]
P. Shor, Algorithms for quantum computation: discrete logarithms and factoring, inProceedings 35th Annual Symposium on Foundations of Computer Science(1994) pp. 124–134
work page 1994
-
[10]
L. K. Grover, A fast quantum mechanical algorithm for database search, inProceedings of the Twenty-Eighth An- nual ACM Symposium on Theory of Computing(1996) p. 212–219
work page 1996
-
[11]
D. S. Abrams and S. Lloyd, Quantum algorithm provid- ing exponential speed increase for finding eigenvalues and eigenvectors, Phys. Rev. Lett.83, 5162 (1999)
work page 1999
-
[12]
A. W. Harrow, A. Hassidim, and S. Lloyd, Quantum al- gorithm for linear systems of equations, Phys. Rev. Lett. 103, 150502 (2009)
work page 2009
-
[13]
D. Kielpinski, C. Monroe, and D. Wineland, Architecture for a large-scale ion-trap quantum computer, Nature417, 709–711 (2002)
work page 2002
-
[14]
P. Kok, W. J. Munro, K. Nemoto, T. C. Ralph, J. P. Dowling, and G. J. Milburn, Linear optical quantum computing with photonic qubits, Rev. Mod. Phys.79, 135 (2007)
work page 2007
-
[15]
C. D. Bruzewicz, J. Chiaverini, R. McConnell, and J. M. Sage, Trapped-ion quantum computing: Progress and challenges, Appl. Phys. Rev.6, 021314 (2019)
work page 2019
-
[16]
M. Kjaergaard, M. E. Schwartz, J. Braum¨ uller, P. Krantz, J. I.-J. Wang, S. Gustavsson, and W. D. Oliver, Superconducting qubits: Current state of play, Annu. Rev. Condens. Matter Phys.11, 369 (2020)
work page 2020
-
[17]
L. M. K. Vandersypen and I. L. Chuang, NMR techniques for quantum control and computation, Rev. Mod. Phys. 76, 1037 (2005)
work page 2005
- [18]
- [19]
-
[20]
A. D. C´ orcoleset al., Challenges and opportunities of near-term quantum computing systems, Proceedings of the IEEE108, 1338 (2020)
work page 2020
-
[21]
T. Proctor, K. Young, A. D. Baczewski,et al., Bench- marking quantum computers, Nat. Rev. Phys.7, 105–118 (2025)
work page 2025
-
[22]
C. McGeoch, How NOT to fool the masses when giv- ing performance results for quantum computers, preprint arXiv:2411.08860 (2024)
-
[23]
T. Proctor, K. Rudinger, K. Young,et al., Measuring the capabilities of quantum computers, Nat. Phys.18, 75–79 (2021)
work page 2021
-
[24]
E. Pelofske, A. B¨ artschi, and S. Eidenbenz, Quantum volume in practice: What users can expect from NISQ devices, IEEE Transactions on Quantum Engineering3, 1 (2022)
work page 2022
-
[25]
C. H. Baldwin, K. Mayer, N. C. Brown, C. Ryan- Anderson, and D. Hayes, Re-examining the quantum vol- ume test: Ideal distributions, compiler optimizations, confidence intervals, and scalable resource estimations, Quantum6, 707 (2022)
work page 2022
-
[26]
L. S. Bishop, S. Bravyi, A. W. Cross, J. M. Gambetta, and J. A. Smolin, Quantum volume, Tech. Rep. (2017)
work page 2017
-
[27]
A. W. Cross, L. S. Bishop, S. Sheldon, P. D. Nation, and J. M. Gambetta, Validating quantum computers using randomized model circuits, Phys. Rev. A100, 032328 (2019)
work page 2019
-
[28]
N. Sundaresan, I. Lauer, E. Pritchett, E. Magesan, P. Ju- rcevic, and J. M. Gambetta, Reducing unitary and spec- tator errors in cross resonance with optimized rotary echoes, PRX Quantum1, 020318 (2020)
work page 2020
-
[29]
J.-S. Chen, E. Nielsen, M. Ebert, V. Inlek, K. Wright, V. Chaplin, A. Maksymov, E. P´ aez, A. Poudel, P. Maunz, and J. Gamble, Benchmarking a trapped-ion quantum computer with 30 qubits, Quantum8, 1516 (2024)
work page 2024
- [30]
-
[31]
A. Cornelissen, J. Bausch, and A. Gily´ en, Scalable benchmarks for gate-based quantum computers, preprint arXiv:2104.10698 (2021)
-
[32]
Castelvecchi, IBM releases first-ever 1000-qubit quan- tum chip, Nature624, 238 (2023)
D. Castelvecchi, IBM releases first-ever 1000-qubit quan- tum chip, Nature624, 238 (2023)
work page 2023
-
[33]
M. AbuGhanem, IBM Quantum Computers: Evo- lution, Performance, and Future Directions, preprint arXiv:2410.00916 (2024)
-
[34]
Google Quantum AI and Collaborators, Quantum er- ror correction below the surface code threshold, Nature (2024)
work page 2024
- [35]
-
[36]
R. Jozsa and A. Miyake, Matchgates and classical simu- lation of quantum circuits, Proceedings of the Royal Soci- ety A: Mathematical, Physical and Engineering Sciences 464, 3089 (2008)
work page 2008
-
[37]
S. Aaronson and L. Chen, Complexity-theoretic founda- tions of quantum supremacy experiments, Cybersecurity and Cyberforensics Conference , Riga Latvia (2016)
work page 2016
-
[38]
E. Bernstein and U. Vazirani, Quantum complexity the- ory, SIAM Journal on Computing26, 1411–1473 (1997)
work page 1997
-
[39]
N. Khaneja, R. Brockett, and S. J. Glaser, Time optimal control in spin systems, Phys. Rev. A63, 032308 (2001)
work page 2001
-
[40]
Liu, Spectral form factors and late time quantum chaos, Phys
J. Liu, Spectral form factors and late time quantum chaos, Phys. Rev. D98, 086026 (2018)
work page 2018
-
[41]
N. S. S. de Buruaga, R. Bistro´ n, M. Rudzi´ nski, R. M. C. 9 Pereira, K. ˙Zyczkowski, and P. Ribeiro, Fidelity decay and error accumulation in random quantum circuits, Sci- Post Phys.19, 013 (2025)
work page 2025
-
[42]
Qiskit documentation,https://docs.quantum.ibm.com
-
[43]
Bistro´ n, GitHub Repository Quantum Volume
R. Bistro´ n, GitHub Repository Quantum Volume
-
[44]
IBM Quantum Platform,https://quantum.ibm.com, ac- cessed: 2024-11-22
work page 2024
-
[45]
C. Gidney and M. Eker˚ a, How to factor 2048 bit RSA in- tegers in 8 hours using 20 million noisy qubits, Quantum 5, 433 (2021)
work page 2048
-
[46]
J. Alman and V. V. Williams, A refined laser method and faster matrix multiplication, TheoretiCSVolume 3, 21 (2024)
work page 2024
-
[47]
J. Hines, M. Lu, R. K. Naik, A. Hashim, J.-L. Ville, B. Mitchell, J. M. Kriekebaum, D. I. Santiago, S. Seri- tan, E. Nielsen, R. Blume-Kohout, K. Young, I. Siddiqi, B. Whaley, and T. Proctor, Demonstrating scalable ran- domized benchmarking of universal gate sets, Phys. Rev. X13, 041030 (2023)
work page 2023
-
[48]
R. Blume-Kohout and K. C. Young, A volumetric frame- work for quantum computer benchmarks, Quantum4, 362 (2020)
work page 2020
-
[49]
S. Aaronson and D. Gottesman, Improved simulation of stabilizer circuits, Phys. Rev. A70, 052328 (2004)
work page 2004
-
[50]
Or´ us, Tensor networks for complex quantum systems, Nat
R. Or´ us, Tensor networks for complex quantum systems, Nat. Rev. Phys.1, 538–550 (2019)
work page 2019
-
[51]
Quantum Convolutional Neural Networks are Effectively Classically Simulable
P. Bermejo, P. Braccia, M. S. Rudolph, Z. Holmes, L. Cincio, and M. Cerezo, Quantum convolutional neural networks are (effectively) classically simulable, preprint arXiv:2408.12739 (2024)
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[52]
A. Angrisani, A. Schmidhuber, M. S. Rudolph, M. Cerezo, Z. Holmes, and H.-Y. Huang, Classically estimating observables of noiseless quantum circuits, preprint arXiv:2409.01706 (2024)
-
[53]
T. Proctor, K. Rudinger, K. Young, E. Nielsen, and R. Blume-Kohout, Measuring the capabilities of quan- tum computers, Nat. Phys.18, 75–79 (2022)
work page 2022
- [54]
- [55]
-
[56]
T. Proctor, S. Seritan, K. Rudinger, E. Nielsen, R. Blume-Kohout, and K. Young, Scalable randomized benchmarking of quantum computers using mirror cir- cuits, Phys. Rev. Lett.129, 150502 (2022)
work page 2022
-
[57]
B. Kraus and J. I. Cirac, Optimal creation of entangle- ment using a two-qubit gate, Phys. Rev. A63, 062309 (2001)
work page 2001
-
[58]
F. Vatan and C. Williams, Optimal quantum circuits for general two-qubit gates, Phys. Rev. A69, 032315 (2004)
work page 2004
-
[59]
M. Musz, M. Ku´ s, and K.˙Zyczkowski, Unitary quantum gates, perfect entanglers, and unistochastic maps, Phys. Rev. A87, 022111 (2013)
work page 2013
-
[60]
S. Lakshmivarahan, S. K. Dhall, and L. L. Miller, Parallel sorting algorithms (Elsevier, 1984) pp. 295–354
work page 1984
-
[61]
E. R. Canfield, S. Janson, and D. Zeilberger, The Ma- honian probability distribution on words is asymptoti- cally normal, Advances in Applied Mathematics46, 109 (2011), special issue in honor of Dennis Stanton
work page 2011
-
[62]
O. Reardon-Smith, M. Oszmaniec, and K. Korzekwa, Im- proved simulation of quantum circuits dominated by free fermionic operations, Quantum8, 1549 (2024). Appendix A: Circuits with parity preservation, and their analytical counterparts In this Appendix, we briefly reintroduce parity preserving and double parity preserving quantum volume circuits discussed ...
work page 2024
-
[63]
We note that some parts of the below calculations were inspired by [39]
Solvable counterpart of parity quantum volume circuit Now we are prepared to present a modification of the parity-preserving quantum volume circuit which does not substantially affect its behavior but enables us to obtain a close analytical formula for an average heavy output probability in the presence of introduced noise model. We note that some parts o...
-
[64]
Its general formula for the heavy output frequency looks exactly like in the previous case (A3)
Solvable counterpart of double-parity quantum volume circuit Now we consider the circuit with double-parity preservation - parity on two, complementary, randomly selected subsets of qubits. Its general formula for the heavy output frequency looks exactly like in the previous case (A3). The only difference is the diagonal gatesu diag (6) appearing each tim...
-
[65]
Dissipative noise model One may generalize the above calculations to encompass other sorts of errors in numerous ways. The most insightful modification from our point of view is the uncontrolled interaction with the environment in the faulty realization of two-qubit gates. The adjustments in calculations are quite minor. The new formula for heavy output f...
-
[66]
Comparison of analytical and numerical results We finish this chapter by illustrating how well the formulas for heavy output frequency derived using modified circuits suit the simulations of actual proposed circuits. To make the presentation more apparent we investigate simplified, exponential formulas (A10) for single parity circuit, and (A24) for double...
work page 2000
-
[67]
For 6-qubit tests presented in the main text, 50 randomly generated circuits were executed 650 times
Device testing For performing the QV, single parity, and double parity tests we executed corresponding circuits onIBM Brisbane device. For 6-qubit tests presented in the main text, 50 randomly generated circuits were executed 650 times. Heavy output subspaces for all circuits were determined through noiseless classical computation. The heavy output probab...
-
[68]
Simulation of the IBM device was performed withAerSimulator
Device simulation Simulations with rescaled errors on the IBM simulator were performed to test the performance of the proposed tests for different noise scales and to compare their results with those of the Quantum Volume test. Simulation of the IBM device was performed withAerSimulator. Using fake backend we modify error parameters i.e.readout errorfor a...
work page 2000
-
[69]
Discussion The presented simulations show agreement between the proposed measures with the Quantum Volume test. How- ever, as we discussed in the main body of the work and above Appendices, different tests have different sensitivities for various error models. Therefore discrepancies are inevitable in some scenarios. We argue that the proposed tests exhib...
-
[70]
Computing estimator ̃hU for standard QV circuit The equation for the heavy output probability for fixedNandTwith the noise model(Ω z)M z=1 is expressed as ̃hU = ∫C=(U 1,...,UM) dC⟨ΩM(UM . . .Ω1(U1∣0⊗N⟩⟨0⊗N∣U† 1). . . U † M),Π C⟩,(D1) whereCis a random circuit of the size(N, T)that containsTlayers of random subsystem permutations and random two-qubit gates...
-
[71]
2N P(M) 0 −1 2N −1 M−1 ∏ z=1 az
-
[72]
Computing the estimator ̃hm U for parity preserving circuits Using similar notation as in previous section, we can express heavy output probability ̃hm U for a quantum volume circuit preserving parity onmqubits as ̃hm U = ∫Cm=(U1,...,UM) dCm⟨ΩM(UM . . .Ω1(U1∣0⊗N⟩⟨0⊗N∣U† 1). . . U † M),Π Cm⟩.(D6) Based on the scheme description and exponential distribution...
-
[73]
Estimatingh U for a given noise model type(Ω z)M z=1 Having established concise expressions for heavy output frequency in different circuits we may leverage them, in particular scenarios, to obtain an estimator of original heavy output frequency. a. No estimation for general noise channelsΩ z Let us fix Ω 1 =Z ⊗N λ and Ω2, . . . ,ΩM =1 l withZ λ =∣0⟩⟨0∣+e...
-
[74]
2N P(M) 0 −1 2N−1 . Meanwhile, for the Eq. (D10) we get ̃hm U = 1 2 ∑ p tr((Π m,p⊗1 l⊗N−m 2 )Ω(s) M (σm,p⊗ρ⊗N−m ∗ )) = 1 2( N m) ∑ p,{q1,...,qm}⊂{1,...,N} tr(πq1,...,qm(Πm,p⊗1 l⊗N−m 2 )π † q1,...,qmΩM(πq1,...,qm(σm,p⊗ρ⊗N−m ∗ )π † q1,...,qm)) = 1 2N( N m) ∑ p,q1,...,qm ∑ i,j∶⊕m k=1iqk=p,⊕m k=1jqk=p tr(∣i⟩⟨i∣ΩM(∣j⟩⟨j∣)) = 1 2N( N m) ∑ p,q1,...,qm ∑ i,j w(M)...
-
[75]
2N⟨v ,̃h∗ U⟩−1 2N −1 .(D14) d. Precise estimation for the maximally depolarizing noises To calculate the estimator ̃hU for depolarizing noise we notice that⟨J Φϵ , J1 l⟩=4−3ϵwhich givesa z = (4−3ϵ)N−1 4N−1 . Moreover,P (M) 0 =(1−ϵ/2) N that gives ̃hU = 1 2 +(p ∗−1
-
[76]
(2−ϵ)N −1 2N −1 ( (4−3ϵ)N −1 4N −1 ) M−1 .(D15) To calculate ̃hm U we notice that tr((Πm,pz ⊗1 l⊗N−m 2 )Ω(s) z (σm,pz−1⊗ρ⊗N−m ∗ )) =tr (Πm,pzΦ⊗m ϵ (σm,pz−1)) = 1 2m−1∑ i,j tr(∣i⟩⟨i∣Φ⊗m ϵ (∣j⟩⟨j∣))δ⊕j=pz−1δ⊕i=pz = 1 2m−1 m ∑ d=0 ∑ i,j∶ρH(i,j)=d (ϵ/2) d(1−ϵ/2)m−dδ⊕j=pz−1δ⊕i=pz = m ∑ d=0 ( m d)(ϵ/2) d(1−ϵ/2)m−dδpz⊕d=pz−1= 1+(−1)pz−pz−1(1−ϵ)m 2 . (D16) Using ...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.