REVIEW 3 major objections 6 minor 17 references
The paper claims that a fixed set of nine 3-qubit probe circuits, read with randomized Pauli shadow measurements at 200 shots per circuit, lets an off-the-shelf random forest identify which of ten simulated quantum noise channels corrupted
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 07:42 UTC pith:5ZWRQZV5
load-bearing objection A modest, simulation-only noise classifier with a clean protocol but a missing parameter mapping that blocks reproduction; worth refereeing, not worth citing until the artifact is fixed. the 3 major comments →
Shadow-Based Noise Fingerprinting of Simulated Quantum Noise Models
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The paper's central claim is that a supervised pipeline — nine 3-qubit probe states, randomized Pauli-basis measurements, physics-informed feature extraction, and a tree-ensemble classifier — can identify which of ten predefined noise channels acted on a circuit, using only 200 shots per probe. Evaluated on 14,000 simulated samples with 1,400 per class, the random forest reaches a mean test accuracy of 0.8426 and macro F1 of 0.8437. Confusion analysis shows most errors are concentrated between channels sharing decay mechanisms, e.g., phase damping vs. thermal relaxation and phase-amplitude damping vs. reset noise, which the paper reads as evidence that the probe/observable set, not classifie
What carries the argument
The central object is the 279-dimensional feature vector. For each of nine 3-qubit probe circuits, 18 raw Pauli expectation values (nine single-qubit and nine two-qubit correlations) are estimated via classical shadow tomography — a randomized measurement protocol that recovers many expectation values from a compact set of shots — and 13 derived quantities are added, including axis means, coherence and population strengths, pairwise differences, and ratio features designed to amplify distinctions between physically similar channels. The probe set combines five parameterized entangled states with four simple structured states, and tree ensembles use the resulting vector to assign a noise clas
Load-bearing premise
The central claim depends on the ten simulated noise channels, sampled with the paper's model-specific parameter mappings, being physically representative and comparably difficult to tell apart — and the paper concedes that its simulations do not capture hardware-specific effects, so this transferability remains an assumption.
What would settle it
Re-run the published dataset-generation code and compute the average gate fidelity or noise strength of each sampled class; if some classes are trivially separable (near-identity for some, fully depolarizing for others), the 84% accuracy is an artifact of dataset construction rather than the features. Alternatively, train on the simulated data and test on noise injected via pulse-level control on a real three-qubit device, with ground truth from process tomography; if accuracy falls to chance, the transfer premise fails.
If this is right
- A device operator could classify its dominant noise channel from nine fixed 3-qubit probe circuits and 200 shots per circuit, sidestepping process tomography's exponential cost.
- The method separates many channel families with high per-class reliability, making it usable as a low-cost first step before choosing an error-mitigation strategy.
- Confusion patterns identify the physical bottleneck — channels paired by shared decay mechanisms — directing future work to richer probes and observables rather than more powerful classifiers.
- Because tree ensembles beat a neural baseline at default hyperparameters, the feature vector, not classifier expressivity, carries the discrimination.
- Scaling analysis shows usable signal well above chance even at 300 samples per class, so the framework does not require huge training sets to be useful.
Where Pith is reading between the lines
- One extension the paper leaves implicit: the same probes and features could estimate continuous noise-strength parameters (e.g., decay rates) instead of discrete labels, by regressing on the derived features.
- The probe set is small enough that a device-wide spatial noise map could be built by running the nine circuits on overlapping 3-qubit neighbourhoods of a larger chip; the paper only evaluates each sample independently.
- The three constant features from the unpopulated mixed-axis group suggest that adding genuine mixed-Pauli observables could reduce the phase-damping/thermal-relaxation confusion; the paper notes the redundancy but does not test this.
- A hardware-robust variant might include calibration drift and correlated noise in training, since the paper's own limitation section acknowledges these are absent from simulation.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a noise-fingerprinting pipeline that combines classical shadow tomography on a fixed set of 3-qubit probe circuits (five random-parameter QAOA circuits and four simple states) with a 279-dimensional feature vector (Pauli expectations plus hand-derived coherence/population/ratio features), followed by supervised classification into one of ten Qiskit noise models. On 14,000 simulated samples (1,400 per class), the random forest classifier is reported to achieve 0.8426 test accuracy and 0.8437 macro F1 on a stratified 20% held-out set, averaged over three seeds; extra trees is close and the MLP lower. Confusion concentrates between physically similar channel pairs. The authors explicitly limit their claims to simulation and to discrete noise-model classification, not noise-parameter estimation.
Significance. If the reported result is robust, the pipeline is a measurement-efficient alternative to full process tomography for routine noise-model identification, requiring only 200 shots per probe circuit. The manuscript has clear strengths: a fixed probe design, a standard stratified split, default scikit-learn hyperparameters for the ensemble classifiers, low variance across three seeds, and a candid limitations section. It also includes confusion and scaling analyses. However, the central empirical claim is conditional on dataset-generation details that are not fully disclosed, and the contribution attributed to physics-informed features is not isolated by ablation. These gaps currently prevent the reader from attributing the 0.8426 accuracy specifically to the proposed method rather than to dataset construction or feature selection.
major comments (3)
- [Sec. IV.A (Dataset) and Sec. I (artifact URL)] The central result is conditional on 'model-specific parameter mappings' that are not in the manuscript and cannot be inspected because the repository URL in Sec. I contains a space (https://github.com/AdaHgrace/Noise Fingerprinting). A uniform p in [0.01,0.15] does not imply comparable physical noise strength across Qiskit models with different parameter semantics (e.g., depolarizing probability vs. amplitude-damping gamma vs. thermal-relaxation T1/T2). If some classes are near-identity over the sampled range while others are strongly decohering, the accuracy could reflect trivial signal-strength separation rather than the discriminative power of the shadow features. Please include the mappings in the paper or appendix, report per-class average process fidelity or another distance metric over the sampled range, and consider a control experiment with normalized per-class strengths or an
- [Sec. III.C and Table I] The abstract and contributions credit 'physics-informed feature engineering' with resolving physically similar noise channels, but the paper contains no ablation comparing the raw 18 Pauli-shadow observables per probe against the full 31-feature set including the 13 derived quantities. Fig. 2 shows that phase damping vs. thermal relaxation and phase-amplitude damping vs. reset noise remain the main confusions, yet no analysis shows which features help or hurt. Add an ablation (raw observables only vs. raw+derived), report feature-importance or permutation-importance results, and discuss whether the handcrafted ratios actually resolve any specific confusion pair. Without this, the role of the claimed physics-informed design is not established.
- [Sec. IV.B / Table I] The manuscript states that random forest 'outperforms' extra trees, but the reported means and standard deviations (0.8426±0.0036 vs. 0.8406±0.0019 over three seeds) make the difference comparable to the combined standard error. Either report paired per-seed results with a proper significance test, or soften the claim to 'comparable performance.' This does not affect the main accuracy claim, but it affects the comparison narrative in the abstract and conclusion.
minor comments (6)
- [Sec. III.C] The ratio features introduce an undefined ε. Specify whether ε is a small constant, its value, and whether it is fixed across all features or tuned. This is needed for reproducibility.
- [Sec. III.B / III.C] The classical shadow reconstruction step is described only verbally. Give the explicit estimator used to convert randomized Pauli bitstrings into the reported expectation values (e.g., the inverted local Clifford channel), including the handling of the 200-shot budget and any median-of-means procedure.
- [Sec. III.C] The population-strength feature Z̄ is stated to duplicate the Z-axis mean and is 'retained to match the evaluated implementation.' This duplicate feature is redundant by construction; either remove it and state the effective dimensionality or justify its retention with an ablation or comment on collinearity.
- [Sec. V.B] The limitation section notes that three derived features associated with an unpopulated mixed-axis observable group are constant across samples. Which three features are these, and why was the observable group unpopulated? Also, constant features cannot contribute to classification; state this explicitly in the feature description rather than only in limitations.
- [Sec. IV.A] State the Qiskit version and the exact class definitions used for the ten noise models. In particular, clarify how readout error is implemented (before measurement vs. as a classical bit-flip) and whether models such as thermal relaxation combine T1/T2 processes in the standard Qiskit way. This information is needed to reproduce the dataset.
- [General] The repository URL in Sec. I should be corrected to a valid URL and, preferably, accompanied by a DOI or versioned release. The current formatting with a space prevents access.
Circularity Check
No significant circularity: the reported accuracy is an empirical held-out classification result on simulated data, not a quantity derived from its own inputs.
full rationale
The paper's central claim is an empirical benchmark: ten Qiskit noise models are used to generate 14,000 labeled samples, shadow-derived features are extracted, and three classifiers are evaluated on a held-out test set. The labels are injected at simulation time and the test set is held out; classifier hyperparameters are scikit-learn defaults except for the specified MLP architecture. There is no fitted parameter that is later renamed as a prediction, no definitional equation equating the target with an input, and no load-bearing self-citation chain: the references to shadow tomography, QAOA, Qiskit, and prior noise-classification work are external and standard. The paper's own limitations section (Sec. V.B) explicitly disclaims generalization to hardware, which appropriately bounds the transfer claim. The main weakness, as the skeptic notes, is that the per-model parameter mappings are only promised in an artifact repository rather than specified in the paper, so the physical difficulty balance across classes is not fully auditable from the text; however, this is a reproducibility or experimental-control concern, not circularity. The derivation chain is self-contained as an empirical study, so no circular step is exhibited.
Axiom & Free-Parameter Ledger
free parameters (4)
- ε in ratio features
- Per-model noise parameter range =
uniform [0.01, 0.15] (model-specific mappings in repo)
- Shot budget =
200 per probe circuit
- QAOA parameter sampling distribution
axioms (4)
- standard math Classical shadow protocol (Huang et al. 2020) reconstructs Pauli expectation values correctly from randomized measurements
- domain assumption Qiskit noise channels faithfully implement the named mathematical channels (depolarizing, amplitude damping, thermal relaxation, etc.)
- domain assumption The ten selected noise models span the practically relevant NISQ error space
- ad hoc to paper The 13 derived features amplify physically meaningful distinctions between channels
read the original abstract
Accurate noise classification is essential for operating near-term quantum processors, yet existing approaches, such as quantum process tomography, scale exponentially with system size, limiting their practicality for routine calibration. We propose a measurement-efficient noise fingerprinting pipeline that combines structured classical shadow tomography with physics-informed feature engineering to identify noise channels from a fixed set of 3-qubit probe circuits. Each sample is represented by a 279-dimensional feature vector constructed from randomized Pauli measurements and derived observables, designed to resolve physically similar noise channels that produce overlapping signatures under generic measurement sets. We evaluate three classifiers, namely, random forest, extra trees, and a multilayer perceptron, on a dataset of 14,000 labeled samples spanning ten noise models. The random forest classifier achieves the highest test accuracy of 0.8426 with a macro F1 score of 0.8437, outperforming both baselines. Confusion analysis reveals that many noise models are classified with high reliability, with the remaining confusions occurring between channels sharing similar physical decay mechanisms, motivating future work on richer probe states and noise parameter estimation.
Figures
Reference graph
Works this paper leans on
-
[1]
Quantum computing in the NISQ era and beyond,
J. Preskill, “Quantum computing in the NISQ era and beyond,”Quan- tum, vol. 2, p. 79, 2018
2018
-
[2]
M. A. Nielsen and I. L. Chuang,Quantum computation and quantum information. Cambridge University Press, 2010
2010
-
[3]
Predicting many properties of a quantum system from very few measurements,
H.-Y . Huang, R. Kueng, and J. Preskill, “Predicting many properties of a quantum system from very few measurements,”Nature Physics, vol. 16, no. 10, pp. 1050–1057, 2020
2020
-
[4]
A quantum approximate optimization algorithm,
E. Farhi, J. Goldstone, and S. Gutmann, “A quantum approximate optimization algorithm,”arXiv preprint arXiv:1411.4028, 2014
Pith/arXiv arXiv 2014
-
[5]
Shadow tomography of quantum states,
S. Aaronson, “Shadow tomography of quantum states,” inProceedings of the 50th annual ACM SIGACT symposium on theory of computing, 2018, pp. 325–338
2018
-
[6]
Experimental estimation of quantum state properties from classical shadows,
G. Struchalin, Y . A. Zagorovskii, E. Kovlakov, S. Straupe, and S. Kulik, “Experimental estimation of quantum state properties from classical shadows,”PRX Quantum, vol. 2, no. 1, p. 010307, 2021
2021
-
[7]
Learning quantum states from their classical shadows,
H.-Y . Huang, “Learning quantum states from their classical shadows,” Nature Reviews Physics, vol. 4, no. 2, p. 81, 2022
2022
-
[8]
Classical shadows with noise,
D. E. Koh and S. Grewal, “Classical shadows with noise,”Quantum, vol. 6, p. 776, 2022
2022
-
[9]
Scalable and robust randomized benchmarking of quantum processes,
E. Magesan, J. M. Gambetta, and J. Emerson, “Scalable and robust randomized benchmarking of quantum processes,”Physical Review Letters, vol. 106, no. 18, p. 180504, 2011
2011
-
[10]
Learning the noise fingerprint of quantum devices,
S. Martina, L. Buffoni, S. Gherardini, and F. Caruso, “Learning the noise fingerprint of quantum devices,”Quantum Machine Intelligence, vol. 4, no. 1, p. 8, 2022
2022
-
[11]
Machine learning based noise characterization and correction on neutral atoms NISQ devices,
E. Canonici, S. Martina, R. Mengoni, D. Ottaviani, and F. Caruso, “Machine learning based noise characterization and correction on neutral atoms NISQ devices,”Advanced Quantum Technologies, vol. 7, no. 1, p. 2300192, 2024
2024
-
[12]
Deep learning enhanced noise spectroscopy of a spin qubit envi- ronment,
S. Martina, S. Hern ´andez-G´omez, S. Gherardini, F. Caruso, and N. Fab- bri, “Deep learning enhanced noise spectroscopy of a spin qubit envi- ronment,”Machine Learning: Science and Technology, vol. 4, no. 2, p. 02LT01, 2023
2023
-
[13]
Noise classification in three-level quan- tum networks by machine learning,
S. Mukherjee, D. Penna, F. Cirinn `a, M. Paternostro, E. Paladino, G. Falci, and L. Giannelli, “Noise classification in three-level quan- tum networks by machine learning,”Machine Learning: Science and Technology, vol. 5, no. 4, p. 045049, 2024
2024
-
[14]
Hamiltonian-driven shadow tomography of quantum states,
H.-Y . Hu and Y .-Z. You, “Hamiltonian-driven shadow tomography of quantum states,”Physical Review Research, vol. 4, no. 1, p. 013054, 2022
2022
-
[15]
Toward live noise fingerprinting in quantum software engineering,
A. Bensoussan, E. Chachkarova, K. Even-Mendoza, S. Fortz, V . Klimis, and M. R. Mousavi, “Toward live noise fingerprinting in quantum software engineering,”arXiv preprint arXiv:2512.18667, 2025
Pith/arXiv arXiv 2025
-
[16]
Quantum computing with Qiskit,
A. Javadi-Abhari, M. Treinish, K. Krsulich, C. J. Wood, J. Lishman, J. Gacon, S. Martiel, P. D. Nation, L. S. Bishop, A. W. Crosset al., “Quantum computing with Qiskit,”arXiv preprint arXiv:2405.08810, 2024
Pith/arXiv arXiv 2024
-
[17]
Scikit-learn: Machine learning in Python,
F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V . Dubourget al., “Scikit-learn: Machine learning in Python,”Journal of Machine Learn- ing Research, vol. 12, pp. 2825–2830, 2011
2011
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.