Pith. sign in

REVIEW 3 major objections 6 minor 17 references

The paper claims that a fixed set of nine 3-qubit probe circuits, read with randomized Pauli shadow measurements at 200 shots per circuit, lets an off-the-shelf random forest identify which of ten simulated quantum noise channels corrupted

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · deepseek-v4-flash

2026-08-02 07:42 UTC pith:5ZWRQZV5

load-bearing objection A modest, simulation-only noise classifier with a clean protocol but a missing parameter mapping that blocks reproduction; worth refereeing, not worth citing until the artifact is fixed. the 3 major comments →

arxiv 2607.08998 v2 pith:5ZWRQZV5 submitted 2026-07-10 cs.SE quant-ph

Shadow-Based Noise Fingerprinting of Simulated Quantum Noise Models

classification cs.SE quant-ph
keywords quantum noise classificationclassical shadow tomographynoise fingerprintingmachine learningnoise channelsfeature engineeringsupervised classificationnear-term quantum devices
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper is trying to show that noise fingerprinting — identifying which error channel damaged a quantum device — does not require full process tomography. It claims that a fixed set of nine 3-qubit probe circuits, read with randomized Pauli (shadow) measurements and only 200 shots per circuit, yields a 279-dimensional feature vector that lets an off-the-shelf random forest name which of ten simulated noise models acted on the qubits, with 84.26% held-out accuracy. The result matters because it points to a measurement-efficient calibration route for near-term devices, where routine noise identification is currently expensive. The paper's own limitation section concedes that all experiments are simulated and that hardware-specific effects such as calibration drift, crosstalk, and correlated noise are not captured, so the transfer to physical processors is not established.

Core claim

The paper's central claim is that a supervised pipeline — nine 3-qubit probe states, randomized Pauli-basis measurements, physics-informed feature extraction, and a tree-ensemble classifier — can identify which of ten predefined noise channels acted on a circuit, using only 200 shots per probe. Evaluated on 14,000 simulated samples with 1,400 per class, the random forest reaches a mean test accuracy of 0.8426 and macro F1 of 0.8437. Confusion analysis shows most errors are concentrated between channels sharing decay mechanisms, e.g., phase damping vs. thermal relaxation and phase-amplitude damping vs. reset noise, which the paper reads as evidence that the probe/observable set, not classifie

What carries the argument

The central object is the 279-dimensional feature vector. For each of nine 3-qubit probe circuits, 18 raw Pauli expectation values (nine single-qubit and nine two-qubit correlations) are estimated via classical shadow tomography — a randomized measurement protocol that recovers many expectation values from a compact set of shots — and 13 derived quantities are added, including axis means, coherence and population strengths, pairwise differences, and ratio features designed to amplify distinctions between physically similar channels. The probe set combines five parameterized entangled states with four simple structured states, and tree ensembles use the resulting vector to assign a noise clas

Load-bearing premise

The central claim depends on the ten simulated noise channels, sampled with the paper's model-specific parameter mappings, being physically representative and comparably difficult to tell apart — and the paper concedes that its simulations do not capture hardware-specific effects, so this transferability remains an assumption.

What would settle it

Re-run the published dataset-generation code and compute the average gate fidelity or noise strength of each sampled class; if some classes are trivially separable (near-identity for some, fully depolarizing for others), the 84% accuracy is an artifact of dataset construction rather than the features. Alternatively, train on the simulated data and test on noise injected via pulse-level control on a real three-qubit device, with ground truth from process tomography; if accuracy falls to chance, the transfer premise fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A device operator could classify its dominant noise channel from nine fixed 3-qubit probe circuits and 200 shots per circuit, sidestepping process tomography's exponential cost.
  • The method separates many channel families with high per-class reliability, making it usable as a low-cost first step before choosing an error-mitigation strategy.
  • Confusion patterns identify the physical bottleneck — channels paired by shared decay mechanisms — directing future work to richer probes and observables rather than more powerful classifiers.
  • Because tree ensembles beat a neural baseline at default hyperparameters, the feature vector, not classifier expressivity, carries the discrimination.
  • Scaling analysis shows usable signal well above chance even at 300 samples per class, so the framework does not require huge training sets to be useful.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • One extension the paper leaves implicit: the same probes and features could estimate continuous noise-strength parameters (e.g., decay rates) instead of discrete labels, by regressing on the derived features.
  • The probe set is small enough that a device-wide spatial noise map could be built by running the nine circuits on overlapping 3-qubit neighbourhoods of a larger chip; the paper only evaluates each sample independently.
  • The three constant features from the unpopulated mixed-axis group suggest that adding genuine mixed-Pauli observables could reduce the phase-damping/thermal-relaxation confusion; the paper notes the redundancy but does not test this.
  • A hardware-robust variant might include calibration drift and correlated noise in training, since the paper's own limitation section acknowledges these are absent from simulation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a noise-fingerprinting pipeline that combines classical shadow tomography on a fixed set of 3-qubit probe circuits (five random-parameter QAOA circuits and four simple states) with a 279-dimensional feature vector (Pauli expectations plus hand-derived coherence/population/ratio features), followed by supervised classification into one of ten Qiskit noise models. On 14,000 simulated samples (1,400 per class), the random forest classifier is reported to achieve 0.8426 test accuracy and 0.8437 macro F1 on a stratified 20% held-out set, averaged over three seeds; extra trees is close and the MLP lower. Confusion concentrates between physically similar channel pairs. The authors explicitly limit their claims to simulation and to discrete noise-model classification, not noise-parameter estimation.

Significance. If the reported result is robust, the pipeline is a measurement-efficient alternative to full process tomography for routine noise-model identification, requiring only 200 shots per probe circuit. The manuscript has clear strengths: a fixed probe design, a standard stratified split, default scikit-learn hyperparameters for the ensemble classifiers, low variance across three seeds, and a candid limitations section. It also includes confusion and scaling analyses. However, the central empirical claim is conditional on dataset-generation details that are not fully disclosed, and the contribution attributed to physics-informed features is not isolated by ablation. These gaps currently prevent the reader from attributing the 0.8426 accuracy specifically to the proposed method rather than to dataset construction or feature selection.

major comments (3)
  1. [Sec. IV.A (Dataset) and Sec. I (artifact URL)] The central result is conditional on 'model-specific parameter mappings' that are not in the manuscript and cannot be inspected because the repository URL in Sec. I contains a space (https://github.com/AdaHgrace/Noise Fingerprinting). A uniform p in [0.01,0.15] does not imply comparable physical noise strength across Qiskit models with different parameter semantics (e.g., depolarizing probability vs. amplitude-damping gamma vs. thermal-relaxation T1/T2). If some classes are near-identity over the sampled range while others are strongly decohering, the accuracy could reflect trivial signal-strength separation rather than the discriminative power of the shadow features. Please include the mappings in the paper or appendix, report per-class average process fidelity or another distance metric over the sampled range, and consider a control experiment with normalized per-class strengths or an
  2. [Sec. III.C and Table I] The abstract and contributions credit 'physics-informed feature engineering' with resolving physically similar noise channels, but the paper contains no ablation comparing the raw 18 Pauli-shadow observables per probe against the full 31-feature set including the 13 derived quantities. Fig. 2 shows that phase damping vs. thermal relaxation and phase-amplitude damping vs. reset noise remain the main confusions, yet no analysis shows which features help or hurt. Add an ablation (raw observables only vs. raw+derived), report feature-importance or permutation-importance results, and discuss whether the handcrafted ratios actually resolve any specific confusion pair. Without this, the role of the claimed physics-informed design is not established.
  3. [Sec. IV.B / Table I] The manuscript states that random forest 'outperforms' extra trees, but the reported means and standard deviations (0.8426±0.0036 vs. 0.8406±0.0019 over three seeds) make the difference comparable to the combined standard error. Either report paired per-seed results with a proper significance test, or soften the claim to 'comparable performance.' This does not affect the main accuracy claim, but it affects the comparison narrative in the abstract and conclusion.
minor comments (6)
  1. [Sec. III.C] The ratio features introduce an undefined ε. Specify whether ε is a small constant, its value, and whether it is fixed across all features or tuned. This is needed for reproducibility.
  2. [Sec. III.B / III.C] The classical shadow reconstruction step is described only verbally. Give the explicit estimator used to convert randomized Pauli bitstrings into the reported expectation values (e.g., the inverted local Clifford channel), including the handling of the 200-shot budget and any median-of-means procedure.
  3. [Sec. III.C] The population-strength feature Z̄ is stated to duplicate the Z-axis mean and is 'retained to match the evaluated implementation.' This duplicate feature is redundant by construction; either remove it and state the effective dimensionality or justify its retention with an ablation or comment on collinearity.
  4. [Sec. V.B] The limitation section notes that three derived features associated with an unpopulated mixed-axis observable group are constant across samples. Which three features are these, and why was the observable group unpopulated? Also, constant features cannot contribute to classification; state this explicitly in the feature description rather than only in limitations.
  5. [Sec. IV.A] State the Qiskit version and the exact class definitions used for the ten noise models. In particular, clarify how readout error is implemented (before measurement vs. as a classical bit-flip) and whether models such as thermal relaxation combine T1/T2 processes in the standard Qiskit way. This information is needed to reproduce the dataset.
  6. [General] The repository URL in Sec. I should be corrected to a valid URL and, preferably, accompanied by a DOI or versioned release. The current formatting with a space prevents access.

Circularity Check

0 steps flagged

No significant circularity: the reported accuracy is an empirical held-out classification result on simulated data, not a quantity derived from its own inputs.

full rationale

The paper's central claim is an empirical benchmark: ten Qiskit noise models are used to generate 14,000 labeled samples, shadow-derived features are extracted, and three classifiers are evaluated on a held-out test set. The labels are injected at simulation time and the test set is held out; classifier hyperparameters are scikit-learn defaults except for the specified MLP architecture. There is no fitted parameter that is later renamed as a prediction, no definitional equation equating the target with an input, and no load-bearing self-citation chain: the references to shadow tomography, QAOA, Qiskit, and prior noise-classification work are external and standard. The paper's own limitations section (Sec. V.B) explicitly disclaims generalization to hardware, which appropriately bounds the transfer claim. The main weakness, as the skeptic notes, is that the per-model parameter mappings are only promised in an artifact repository rather than specified in the paper, so the physical difficulty balance across classes is not fully auditable from the text; however, this is a reproducibility or experimental-control concern, not circularity. The derivation chain is self-contained as an empirical study, so no circular step is exhibited.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central claim rests on three classes of input: (i) standard shadow-tomography math (cited, not re-derived); (ii) the assumption that Qiskit simulations faithfully represent the ten named noise channels with comparably difficult parameters; (iii) a feature design whose discriminative value is asserted but not ablated. No constants were fitted to the test result; the main uncontrolled choices are the unseen per-model parameter mappings and the 200-shot budget.

free parameters (4)
  • ε in ratio features
    The four ratio features add ε to denominators (Sec. III.C) to avoid division by zero; its value is not reported and changes feature scale.
  • Per-model noise parameter range = uniform [0.01, 0.15] (model-specific mappings in repo)
    Data-generation choice (Sec. IV.A); controls class difficulty and is load-bearing for the accuracy number; the mappings are not in the paper.
  • Shot budget = 200 per probe circuit
    The measurement-efficiency claim rests on this budget; no sensitivity analysis is provided.
  • QAOA parameter sampling distribution
    Sec. III.B samples ansatz parameters randomly 'to ensure diversity' without stating the distribution; affects probe-state coverage and dataset variability.
axioms (4)
  • standard math Classical shadow protocol (Huang et al. 2020) reconstructs Pauli expectation values correctly from randomized measurements
    Invoked in Sec. III.C as the basis for the first 18 features per probe; the paper relies on this without re-deriving it.
  • domain assumption Qiskit noise channels faithfully implement the named mathematical channels (depolarizing, amplitude damping, thermal relaxation, etc.)
    The dataset labels are generated by these simulations (Secs. II.A, IV.A); if the simulator misrepresents the physical channels, all labels are corrupted.
  • domain assumption The ten selected noise models span the practically relevant NISQ error space
    Sec. II.A asserts breadth; the classifier only discriminates among these ten, so coverage of real device noise is an unvalidated premise.
  • ad hoc to paper The 13 derived features amplify physically meaningful distinctions between channels
    Sec. III.C motivates ratios like (X̄+Ȳ)/(Z̄+ε) informally; the paper provides no ablation or theorem showing discriminative gain, and Fig. 2 shows the targeted confusions persist.

pith-pipeline@v1.3.0-alltime-deepseek · 143 in / 15513 out tokens · 197051 ms · 2026-08-02T07:42:16.459175+00:00 · methodology

0 comments
read the original abstract

Accurate noise classification is essential for operating near-term quantum processors, yet existing approaches, such as quantum process tomography, scale exponentially with system size, limiting their practicality for routine calibration. We propose a measurement-efficient noise fingerprinting pipeline that combines structured classical shadow tomography with physics-informed feature engineering to identify noise channels from a fixed set of 3-qubit probe circuits. Each sample is represented by a 279-dimensional feature vector constructed from randomized Pauli measurements and derived observables, designed to resolve physically similar noise channels that produce overlapping signatures under generic measurement sets. We evaluate three classifiers, namely, random forest, extra trees, and a multilayer perceptron, on a dataset of 14,000 labeled samples spanning ten noise models. The random forest classifier achieves the highest test accuracy of 0.8426 with a macro F1 score of 0.8437, outperforming both baselines. Confusion analysis reveals that many noise models are classified with high reliability, with the remaining confusions occurring between channels sharing similar physical decay mechanisms, motivating future work on richer probe states and noise parameter estimation.

Figures

Figures reproduced from arXiv: 2607.08998 by Lei Zhang, Vridhi Jain.

Figure 1
Figure 1. Figure 1: Shadow-based noise fingerprinting pipeline consisting of four stages: probe circuit preparation, noisy device execution, randomized shadow [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Confusion matrix for the random forest classifier evaluated on the [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Test accuracy as a function of samples per class for random forest, [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 3
Figure 3. Figure 3: Test accuracy as a function of samples per class for random forest, [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

17 extracted references · 3 linked inside Pith

  1. [1]

    Quantum computing in the NISQ era and beyond,

    J. Preskill, “Quantum computing in the NISQ era and beyond,”Quan- tum, vol. 2, p. 79, 2018

  2. [2]

    M. A. Nielsen and I. L. Chuang,Quantum computation and quantum information. Cambridge University Press, 2010

  3. [3]

    Predicting many properties of a quantum system from very few measurements,

    H.-Y . Huang, R. Kueng, and J. Preskill, “Predicting many properties of a quantum system from very few measurements,”Nature Physics, vol. 16, no. 10, pp. 1050–1057, 2020

  4. [4]

    A quantum approximate optimization algorithm,

    E. Farhi, J. Goldstone, and S. Gutmann, “A quantum approximate optimization algorithm,”arXiv preprint arXiv:1411.4028, 2014

  5. [5]

    Shadow tomography of quantum states,

    S. Aaronson, “Shadow tomography of quantum states,” inProceedings of the 50th annual ACM SIGACT symposium on theory of computing, 2018, pp. 325–338

  6. [6]

    Experimental estimation of quantum state properties from classical shadows,

    G. Struchalin, Y . A. Zagorovskii, E. Kovlakov, S. Straupe, and S. Kulik, “Experimental estimation of quantum state properties from classical shadows,”PRX Quantum, vol. 2, no. 1, p. 010307, 2021

  7. [7]

    Learning quantum states from their classical shadows,

    H.-Y . Huang, “Learning quantum states from their classical shadows,” Nature Reviews Physics, vol. 4, no. 2, p. 81, 2022

  8. [8]

    Classical shadows with noise,

    D. E. Koh and S. Grewal, “Classical shadows with noise,”Quantum, vol. 6, p. 776, 2022

  9. [9]

    Scalable and robust randomized benchmarking of quantum processes,

    E. Magesan, J. M. Gambetta, and J. Emerson, “Scalable and robust randomized benchmarking of quantum processes,”Physical Review Letters, vol. 106, no. 18, p. 180504, 2011

  10. [10]

    Learning the noise fingerprint of quantum devices,

    S. Martina, L. Buffoni, S. Gherardini, and F. Caruso, “Learning the noise fingerprint of quantum devices,”Quantum Machine Intelligence, vol. 4, no. 1, p. 8, 2022

  11. [11]

    Machine learning based noise characterization and correction on neutral atoms NISQ devices,

    E. Canonici, S. Martina, R. Mengoni, D. Ottaviani, and F. Caruso, “Machine learning based noise characterization and correction on neutral atoms NISQ devices,”Advanced Quantum Technologies, vol. 7, no. 1, p. 2300192, 2024

  12. [12]

    Deep learning enhanced noise spectroscopy of a spin qubit envi- ronment,

    S. Martina, S. Hern ´andez-G´omez, S. Gherardini, F. Caruso, and N. Fab- bri, “Deep learning enhanced noise spectroscopy of a spin qubit envi- ronment,”Machine Learning: Science and Technology, vol. 4, no. 2, p. 02LT01, 2023

  13. [13]

    Noise classification in three-level quan- tum networks by machine learning,

    S. Mukherjee, D. Penna, F. Cirinn `a, M. Paternostro, E. Paladino, G. Falci, and L. Giannelli, “Noise classification in three-level quan- tum networks by machine learning,”Machine Learning: Science and Technology, vol. 5, no. 4, p. 045049, 2024

  14. [14]

    Hamiltonian-driven shadow tomography of quantum states,

    H.-Y . Hu and Y .-Z. You, “Hamiltonian-driven shadow tomography of quantum states,”Physical Review Research, vol. 4, no. 1, p. 013054, 2022

  15. [15]

    Toward live noise fingerprinting in quantum software engineering,

    A. Bensoussan, E. Chachkarova, K. Even-Mendoza, S. Fortz, V . Klimis, and M. R. Mousavi, “Toward live noise fingerprinting in quantum software engineering,”arXiv preprint arXiv:2512.18667, 2025

  16. [16]

    Quantum computing with Qiskit,

    A. Javadi-Abhari, M. Treinish, K. Krsulich, C. J. Wood, J. Lishman, J. Gacon, S. Martiel, P. D. Nation, L. S. Bishop, A. W. Crosset al., “Quantum computing with Qiskit,”arXiv preprint arXiv:2405.08810, 2024

  17. [17]

    Scikit-learn: Machine learning in Python,

    F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V . Dubourget al., “Scikit-learn: Machine learning in Python,”Journal of Machine Learn- ing Research, vol. 12, pp. 2825–2830, 2011