Non-Clifford Benchmarking via Ensemble Feature Selection
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-02 11:34 UTCgrok-4.3pith:YAI77CYZrecord.jsonopen to challenge →
The pith
Ensemble Feature Selection trains a linear estimator on simulated noisy channels to measure process infidelity of non-Clifford gates such as CCZ.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The EFS method selects a compact set of circuit measurements from a candidate pool through offline training on an ensemble of noisy channels and combines them into a linear estimator with weights learned by ridge regression, enabling fast estimation of process infidelity for non-Clifford targets.
What carries the argument
Ensemble Feature Selection (EFS), which uses ridge regression on a training ensemble of noisy channels to learn a linear combination of measurement outcomes for estimating gate infidelity.
If this is right
- EFS achieves close agreement with IRB on validation benchmarks across a range of infidelities.
- The estimator provides precision of approximately 0.01 over process infidelity 0.02-0.2.
- EFS can be applied directly to estimate infidelity of the CCZ gate on the device.
- The training ensemble can incorporate prior knowledge about hardware noise mechanisms.
Where Pith is reading between the lines
- If the training ensemble is tuned well, EFS could reduce the number of experiments needed compared to full randomized benchmarking for non-Clifford gates.
- This approach might extend to other non-Clifford gates beyond CCZ if similar structurally related validation benchmarks exist.
- Validation on multiple devices would test how well the method generalizes beyond the single IBM device tested.
Load-bearing premise
The training ensemble of noisy channels must be representative enough of the real device's dominant noise so that the learned estimator generalizes accurately to actual measurements.
What would settle it
A significant discrepancy between EFS estimates and independent IRB results on the Clifford validation benchmarks would indicate the method does not work as claimed.
Figures
read the original abstract
We propose an Ensemble Feature Selection (EFS) method for fast estimation of process infidelity of involutory multi-qubit gates, including non-Clifford targets, for which standard Clifford-based benchmarking does not apply. The method selects a compact set of experimentally executable circuit measurements from a candidate pool through offline training on a physically motivated ensemble of noisy channels, and combines them into a linear estimator with weights learned by ridge regression. The training ensemble is an explicit and tunable component of the protocol, incorporating prior knowledge about dominant hardware noise mechanisms. The estimator is validated on ibm_kingston using two Clifford validation benchmarks structurally related to the transpiled CCZ circuit, against independent Interleaved Randomized Benchmarking (IRB). Both show close EFS-IRB agreement across a wide range of process infidelities, with an estimation precision of approximately 0.01 over a process infidelity range of 0.02-0.2. EFS is subsequently applied directly to CCZ on the same device.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes an Ensemble Feature Selection (EFS) protocol for estimating process infidelity of involutory multi-qubit gates, including non-Clifford targets. A compact set of executable circuit measurements is selected from a candidate pool by offline ridge-regression training on an explicit ensemble of noisy channels that incorporates prior knowledge of dominant hardware noise; the resulting linear estimator is validated on ibm_kingston against independent IRB using two Clifford benchmarks structurally related to a transpiled CCZ, reported to agree within ~0.01 precision over infidelity range 0.02–0.2, and then applied directly to the CCZ gate on the same device.
Significance. If the learned estimator generalizes from the training ensemble to real-device non-Clifford gates, the method would supply a practical, low-shot alternative to full tomography or Clifford-restricted benchmarking for involutory non-Clifford operations. The explicit, tunable character of the training ensemble is a methodological strength that allows incorporation of device-specific noise priors.
major comments (2)
- [Abstract] Abstract: validation is performed exclusively on two Clifford circuits that remain Clifford after transpilation; the central claim that the EFS estimator applies to the non-Clifford CCZ therefore rests on the untested assumption that the chosen ensemble is representative of the device noise for the target gate. No independent ground-truth comparison (IRB or otherwise) is reported for the CCZ itself.
- [Abstract] Abstract: the reported estimation precision of approximately 0.01 is stated without accompanying details on ensemble construction, ridge-regularization parameter selection, checks for overfitting, or error propagation; these omissions are load-bearing for evaluating whether the quoted precision is robust or an artifact of the training procedure.
minor comments (1)
- [Abstract] The abstract refers to “involutory multi-qubit gates” without specifying the precise algebraic condition (e.g., U² = I) used to define the class; a short clarifying sentence would improve scope precision.
Simulated Author's Rebuttal
We thank the referee for the careful review and constructive feedback. We address the two major comments point by point below. We agree that the abstract requires additional methodological details and will revise it. Regarding validation of the non-Clifford target, we clarify the rationale for the chosen benchmarks while acknowledging the indirect nature of the evidence.
read point-by-point responses
-
Referee: [Abstract] Abstract: validation is performed exclusively on two Clifford circuits that remain Clifford after transpilation; the central claim that the EFS estimator applies to the non-Clifford CCZ therefore rests on the untested assumption that the chosen ensemble is representative of the device noise for the target gate. No independent ground-truth comparison (IRB or otherwise) is reported for the CCZ itself.
Authors: We agree that no direct IRB comparison is possible for the CCZ, as IRB requires Clifford gates by construction; this is precisely the motivation for developing EFS. The two validation benchmarks were chosen because they are structurally related to the transpiled CCZ (same qubit layout and similar gate decomposition) and remain Clifford after transpilation, allowing independent IRB. The training ensemble is explicitly constructed from an ensemble of noisy channels that incorporates measured device noise priors (e.g., dominant depolarizing and coherent error channels on ibm_kingston). We will add a new paragraph in the revised manuscript (Section 5) explicitly stating the assumption of ensemble representativeness, its justification from the device-specific training, and the limitation that direct ground-truth validation for non-Clifford gates is unavailable with current methods. revision: partial
-
Referee: [Abstract] Abstract: the reported estimation precision of approximately 0.01 is stated without accompanying details on ensemble construction, ridge-regularization parameter selection, checks for overfitting, or error propagation; these omissions are load-bearing for evaluating whether the quoted precision is robust or an artifact of the training procedure.
Authors: We accept this criticism. The abstract currently omits these details. In the full manuscript, ensemble construction is described in Section 3 (candidate pool of 200 circuits, reduced to 12 via feature selection), the ridge parameter λ is chosen by 5-fold cross-validation on the training ensemble (Section 4.2), overfitting is checked via hold-out validation on a separate noise realization set with reported R^{2} > 0.95, and precision is obtained via bootstrap resampling of the linear estimator (1000 resamples) yielding a standard error of ~0.01. We will revise the abstract to include a concise sentence summarizing these elements and will ensure the methods section makes the full procedure reproducible. revision: yes
Circularity Check
No circularity: estimator trained offline on independent simulated ensemble
full rationale
The EFS protocol trains the linear estimator via ridge regression on an explicit simulated ensemble of noisy channels chosen to reflect hardware noise, then applies the fixed weights to real-device measurement outcomes on the target gate. Validation against IRB occurs on separate Clifford circuits using independent hardware data. No equation or step reduces the reported infidelity for CCZ to a quantity fitted from the CCZ measurements themselves; the training data and target data are disjoint by construction. This matches the default expectation of a self-contained protocol with no load-bearing self-definition or fitted-input-as-prediction.
Axiom & Free-Parameter Ledger
axioms (2)
- standard math Ridge regression produces stable weights for a linear combination of measurement outcomes that estimate process infidelity
- domain assumption An ensemble of simulated noisy channels can be constructed to capture the dominant error mechanisms present on the target hardware
Reference graph
Works this paper leans on
-
[1]
E. Magesan, J. M. Gambetta, and J. Emerson, Phys. Rev. Lett.106, 180504 (2011)
work page 2011
-
[2]
E. Magesan, J. M. Gambetta, B. R. Johnson, C. A. Ryan, J. M. Chow, S. T. Merkel, M. P. Da Silva, G. A. Keefe, 9 M. B. Rothwell, T. A. Ohki, M. B. Ketchen, and M. Stef- fen, Phys. Rev. Lett.109, 080505 (2012)
work page 2012
-
[3]
A. W. Cross, E. Magesan, L. S. Bishop, J. A. Smolin, and J. M. Gambetta, npj Quantum Inf2, 16012 (2016)
work page 2016
- [4]
- [5]
- [6]
- [7]
-
[8]
M. Papi´ c, A. Auer, and I. de Vega, arXiv preprint arXiv:2305.08916 10.48550/arXiv.2305.08916 (2023)
-
[9]
T. Proctor, A. Tran, X. Liu, A. Dhumuntarao, S. Seritan, A. Green, and N. M. Linke, arXiv:2504.12575 (2025), arXiv:2504.12575
- [10]
-
[11]
V. Bol’on-Canedo and A. Alonso-Betanzos, Inf. Fusion 52, 1 (2019)
work page 2019
-
[12]
A. D. Hill, M. J. Hodson, N. Didier, and M. J. Reagor, arXiv:2108.01652 10.48550/arXiv.2108.01652 (2021), arXiv:2108.01652 [quant-ph]
-
[13]
Y.Kim, A.Morvan, L.B.Nguyen, R.K.Naik, C.J¨ unger, L.Chen, J.M.Kreikebaum, D.I.Santiago,andI.Siddiqi, Nat. Phys.18, 783 (2022)
work page 2022
-
[14]
A. Hashim, L. B. Nguyen, N. Goss, B. Marinelli, R. K. Naik, T. Chistolini, J. Hines, J. P. Marceaux, Y. Kim, P. Gokhale, T. Tomesh, S. Chen, L. Jiang, S. Ferracin, K. Rudinger, T. Proctor, K. C. Young, R. Blume-Kohout, and I. Siddiqi, arXiv:2408.12064 10.48550/arXiv.2408.12064 (2024), arXiv:2408.12064 [quant-ph]
- [15]
-
[16]
J. J. Wallman and J. Emerson, Phys. Rev. A94, 052325 (2016)
work page 2016
-
[17]
J. J. Wallman and J. Emerson, System and methods for local randomized benchmarking, US Patent Application 2019/0026211 A1 (2019), filed Jul. 23, 2018; published Jan. 24, 2019
work page 2019
-
[18]
S. K. Grigorova, N. V. Vitanov, and B. T. Torosov, arXiv 10.48550/arXiv.2606.30327 (2026), 2606.30327 [quant- ph]
work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2606.30327 2026
-
[19]
M. A. Nielsen, Physics Letters A303, 249 (2002)
work page 2002
-
[20]
D. C. McKay, C. J. Wood, S. Sheldon, J. M. Chow, and J. M. Gambetta, Phys. Rev. A96, 022330 (2017)
work page 2017
-
[21]
IBM Quantum, Ibm quantum platform (2026), accessed: 2026-06-06
work page 2026
- [22]
-
[23]
S. Krinner, S. Lazar, A. Remm, C. K. Andersen, N. Lacroix, G. J. Norris, C. Hellings, M. Gabureac, C. Eichler, and A. Wallraff, Phys. Rev. Applied14, 024042 (2020)
work page 2020
-
[24]
A. N. Tikhonov and V. Y. Arsenin,Solutions of Ill-Posed Problems(Winston, 1977)
work page 1977
-
[25]
A. E. Hoerl and R. W. Kennard, Technometrics12, 55 (1970)
work page 1970
- [26]
-
[27]
T. Q. A. D. Team, Qiskit aer,https://github.com/ Qiskit/qiskit-aer(2025)
work page 2025
-
[28]
R. Kohavi, inProceedings of the Fourteenth International Joint Conference on Artificial Intelligence (IJCAI)(Mor- gan Kaufmann, 1995) pp. 1137–1143. 10 Appendix A: Gate Circuits This appendix shows the particular transpiled bench- mark circuits used in the experimental validation of the EFS estimator. These circuits are not unique; many al- ternative real...
work page 1995
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.