{"id":"420e582e-df9f-4724-a1a9-8f88e1968663","arxiv_id":"2411.18715","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Distributions of drifting randomized benchmarking error rates, compared via the Kolmogorov-Smirnov test, can discriminate among qubit noise models that match the same dephasing times.","lead":"This paper shows that the drift in a qubit's randomized benchmarking error rate over repeated runs can be compared with a simple statistical test to rule out candidate noise models. It matters because noise models that match standard one-number metrics can still be told apart through the distribution of drifting error rates, using data experimentalists already collect.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Out-of-family noise not tested: the KS validation is only demonstrated within the Table I Gaussian OU family, so the central 'rule out noise models' claim lacks an out-of-family robustness check.","rationale":"I read the paper in good faith. Within the simulated Table I family, the statistical machinery is careful: type I rates are calibrated per model, type II errors are shown in grids, and Appendices F, G, H, J, and K honestly expose limitations. The central claim, however, has two parts: (1) within a candidate family, KS tests can discriminate models that share T2*, and (2) the method can rule out noise models in practice. Part (1) is demonstrated. Part (2) additionally requires that an experiment generated outside the candidate family does not get falsely accepted as one of the candidates. The paper never tests this, and its own nonlinearity observations (Appendix G, Appendix H) make accidental matches plausible. This is not a disagreement with consensus; it is a correct-identification risk in the method's stated use case. The concrete test is feasible with the authors' simulator and would settle whether the concern lands. Since the reader already judged CONDITIONAL and the requested check is a natural condition, I keep the verdict unchanged rather than escalating.","tokens_in":36666,"tokens_out":4668,"duration_ms":46482,"concrete_test":"Simulate a reference 'experiment' under a deliberately out-of-family noise model matched to the same T2* values: e.g., charge and magnetic noise as sums of random-telegraph (two-level) processes, or as OU processes with unequal powers and spectral exponent alpha=0.8 over the same band. Run the same wall-clock RB simulator and apply the KS validation protocol using the 10 Table I models as the only candidates. Record the minimum KS statistic over candidates and the accept/reject decision at the Appendix F thresholds for many seeds. If any wrong candidate is accepted in a substantial fraction of seeds (comparable to the in-family type II rates in Fig. 3), the 'rule out' claim in its stated form fails; if all wrong candidates are rejected, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (abstract, Sec. III C) is that KS tests on drifting RB error-rate distributions can rule out candidate noise models. The demonstration generates the reference 'experiment' from one of the 10 Gaussian, equal-power, one-OU-per-decade models in Table I, and tests only against the other 9. This is a closed family: null and alternative both live in the same parametric class, so Fig. 3 measures within-family identifiability only. For a real device, the experiment is produced by whatever noise actually exists. If that noise is non-Gaussian (e.g., two-level fluctuators, explicitly deferred in Sec. II B), has unequal OU powers, a different spectral exponent, or a spectral feature outside the Table I cutoffs, nothing guarantees the KS test will reject all wrong candidates. The mapping from noise statistics to RB-rate distributions is nonlinear: Appendices G and H show additivity fails on individual realizations and results depend on an ad hoc frequency partition. An out-of-family process could therefore accidentally match one of the Table I distributions and be falsely accepted. The paper provides no experiment or simulation with a reference outside the candidate family, so the externally load-bearing half of the claim is unverified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes using the distribution of drifting randomized benchmarking (RB) error rates, compared via two-sample Kolmogorov-Smirnov tests, to validate or rule out candidate noise models for a singlet-triplet qubit. The authors model charge and magnetic noise as sums of equal-power Ornstein-Uhlenbeck processes, one per frequency decade, calibrate ten candidate models to the same T2* values, generate full wall-clock RB simulations over 100 passes, and construct type I/II error grids. They find that models with different low-frequency content can be distinguished, while models sharing similar high-frequency content are more difficult to separate. They then use a 'validated' model to attribute RB error to charge versus magnetic axes and to low- versus high-frequency components, emphasizing non-additivity and sensitivity to the chosen frequency partition.","tokens_in":36853,"tokens_out":6752,"duration_ms":64634,"significance":"If the central claim holds, the paper offers a practical way to exploit RB drift data that experimentalists are likely already collecting, going beyond single-number T2* characterization. The statistical design is careful in several respects: per-model rejection thresholds are calibrated to equal type I error; Appendix K proves the monotone type I/II tradeoff; the FID/T2* analytic calculations in Appendix D are clean; RB fit quality and aliasing checks are documented. The demonstration is reproducible from the parameter tables. The main unverified step is that the reference 'experiment' is always drawn from the same Gaussian OU family used for the candidates, so the claim's external validity for out-of-family device noise is not yet established.","major_comments":[{"comment":"The demonstration only tests discrimination within a closed family: the reference 'experiment' is one of the ten Gaussian, equal-power, one-OU-per-decade models in Table I, and the candidates are the other nine. The abstract's claim that the test 'can be used to rule out noise models' is not tested for out-of-family noise (e.g., non-Gaussian two-level fluctuators, unequal OU powers, a different spectral exponent, or spectral features outside the IR/UV cutoffs), even though Sec. II B states that Gaussianity is not essential and that extension to such components is straightforward. Since the mapping from noise statistics to RB-rate distributions is nonlinear and Appendices G and H show additivity failures on individual realizations, an out-of-family process could conceivably match an in-family distribution and be falsely accepted. Please add simulation(s) with a reference outside the candidate family, or explicitly restrict the claim to decisions among a specified candidate family.","section":"Abstract and Sec. III C, Table I"},{"comment":"The error attribution to low- and high-frequency components is built on the ad hoc partition in Eq. (12), chosen at the 1/e point of the survival probability. Appendix H shows that with a different partition (Eq. H1) the additivity at large error rates is not satisfied, so the main-text conclusion that high-frequency noise contributes an approximately constant error level and low-frequency noise dominates worst-case error is partition-dependent. The paper should either derive an objective partition criterion from the protocol or explicitly present the attribution as illustrative for a chosen partition.","section":"Sec. IV B, Eq. (12), Appendix H"},{"comment":"The RB error rate r per pass is a point estimate obtained by fitting survival probabilities from only 10 circuits per depth, and the fit uncertainties are visible in Figs. 7 and 8. These uncertainties are not propagated into the KS statistic, so the 'drift' in the distribution of r is partly contaminated by fit noise. Because reference and candidate datasets are processed identically, the type I calibration is not affected, but the type II error rates in Fig. 3 could overstate discriminating power if fit noise contributes substantially. Please quantify the effect of fit uncertainty, for example by repeating the analysis with more circuits per depth or by propagating parametric fit uncertainties into the KS procedure.","section":"Sec. II C and Appendix E"}],"minor_comments":[{"comment":"The labels 'false negative' and 'false positive' are reversed relative to standard usage: rejecting a true null hypothesis (the definition of α in Eq. (10)) is a false positive, while failing to reject a false null (the definition of β) is a false negative. Please correct the terminology.","section":"Sec. III B"},{"comment":"The sentence 'Since a single RB pass takes approximately 1 s' conflicts with the main text, which states that 100 passes correspond to approximately 0.99 s. The pass duration should be corrected to the millisecond scale or the conflict explained.","section":"Appendix H"},{"comment":"The notation α_{X,j} for the type I error rate and β_{X,jj'} for the type II error rate is clear, but the text could state explicitly that these are empirical rates estimated from a finite number of seeds; this is implied later in Sec. III C but would help the reader.","section":"Sec. III B"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within the journal's scope and I do not see evidence of circularity in the main validation procedure. The main gating issue is the out-of-family robustness of the central claim; the authors should either add such tests or narrow the claim. No citation or novelty concerns arose in my reading."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe useful thing here is the idea: if you run multiple passes of RB on a drifting qubit, the distribution of error rates across passes is a model-validation dataset that carries information beyond T2* or a single RB number. The authors show, with careful statistics, that a simple Kolmogorov-Smirnov test on that distribution can discriminate among a ten-model family of Gaussian OU noise models that all match the same T2* values. The statistical machinery is genuinely careful: per-model rejection thresholds calibrated to equal type I error, type I/II error grids, and a proof of the tradeoff in Appendix K. The FID/T2* derivations in Appendix D are clean. The error-attribution section also makes a real point: additivity of error components fails under coherent correlated noise, and Appendices G and H show this is not a minor artifact. That is a useful caution for anyone doing attribution.\n\nThe main soft spot is the one the stress-test flags. The entire demonstration is a closed family: the reference 'experiment' is generated from one of the ten Table I models and tested only against the other nine. So Fig. 3 shows within-family identifiability, not the ability to rule out arbitrary wrong models. If a real device has two-level fluctuators, a different spectral exponent, or spectral features outside the chosen IR/UV cutoffs, nothing in the paper guarantees the KS test will reject all wrong candidates; an out-of-family process could accidentally match one of the Table I distributions and be falsely accepted. The authors say in Sec. II B that Gaussianity is 'not essential' and the method 'could be extended' to non-Gaussian components, but they do not test that, and the abstract's 'rule out noise models' is broader than the evidence. This is a limitation, not a contradiction: the paper is honestly scoped in most of its text, but the abstract overstates.\n\nTwo smaller concerns. No code or data is shipped, so the numerics cannot be re-run. And the error-attribution partition into low/high frequency is ad hoc—Appendix H shows results change with a different partition, which the authors disclose but do not resolve. Discriminating power is also modest in the hard low-frequency cases (thresholds 0.18 to 0.62), so an experimentalist should know the test will not always be decisive.\n\nWho gets value: anyone doing qubit characterization, especially for spin qubits with 1/f noise, because it makes use of RB data already being collected. The paper deserves a serious referee; I would send it to review and ask for an out-of-family robustness check or, failing that, a clear statement that the method is a within-family consistency test.","headline":"Useful idea and careful statistics, but the headline claim is only demonstrated inside the simulated Gaussian OU family.","tokens_in":37472,"tokens_out":3182,"would_cite":true,"duration_ms":28035,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Drifting RB error rates rule out wrong qubit noise models.","keywords":["randomized benchmarking","qubit noise","model validation","Kolmogorov-Smirnov test","low-frequency noise drift","singlet-triplet qubit","Ornstein-Uhlenbeck process","error attribution"],"falsifier":"Measure the power spectral density of the charge and magnetic noise of a singlet-triplet qubit from $10^{-3}$ Hz to $10^{7}$ Hz and compare it to the equal-power, one-OU-per-decade spectrum with the stated cutoffs and powers; if the measured spectrum deviates beyond what the OU family can represent, or if a non-Gaussian two-level fluctuator is present, the K-S error-rate grid computed from this family will not describe the real device. Alternatively, run the wall-clock RB protocol on a device with a known non-Gaussian noise source and check whether the K-S test still separates the ten Gaussian models: a failure to reject all models would show the Gaussian spectral-shape assumption is essential.","tokens_in":36370,"feed_emoji":"⚛️","tokens_out":9308,"duration_ms":84540,"temperature":0.7,"pith_summary":"The paper shows that when randomized benchmarking (RB) is repeated many times on a qubit, the single reported error rate drifts from pass to pass, and the distribution of these drifting values carries information about the noise spectrum. Using wall-clock simulations of a singlet-triplet qubit with Ornstein-Uhlenbeck noise, it demonstrates that a two-sample Kolmogorov-Smirnov test on the distribution of RB error rates can reject candidate noise models that are indistinguishable by the T2* dephasing time alone. This requires sampling over a long enough time window to capture the low-frequency noise components. Once a model passes the test, error attribution by splitting noise sources is also shown to be non-additive, meaning contributions cannot simply be summed.","feed_headline":"Drifting RB error rates rule out wrong qubit noise models","feed_subtitle":"Repeated benchmarking passes turn qubit drift into a model-validation signal a single T2* number cannot provide.","key_machinery":"The central machinery is wall-clock randomized benchmarking simulation: a single noise trajectory built from independent Gaussian Ornstein-Uhlenbeck processes (one per decade, equal power, set by the measured charge and magnetic $T_2^*$) is maintained across 100 RB passes, and each pass yields an error rate $r$ (or, per circuit, a survival probability). The two-sample Kolmogorov-Smirnov statistic compares the cumulative distributions of these metrics between a reference model and a tested model, with rejection thresholds chosen to give equal type I error rates; the result is an empirical error-rate grid of type I and type II errors. The K-S statistic is the discriminator, the OU family is the hypothesis class, and the experiment duration sets which noise frequencies are actually sampled.","core_discovery":"The central claim is that the distribution of randomized benchmarking (RB) error rates obtained over repeated passes can serve as a validation test for noise models, using a statistic as simple as the two-sample Kolmogorov-Smirnov test. Ten noise models of a singlet-triplet qubit, all matched to the same charge and magnetic $T_2^*$ dephasing times, are simulated in wall-clock RB, and the K-S test on the resulting distributions of per-pass error rates produces an error-rate grid that separates the models. The discriminating power is largest when the experimental window captures the low-frequency components of the noise; models with only low-frequency noise have weak power. The paper further shows that once a model is validated, error attribution by separating noise sources is not simply additive, because different noise components can interfere and partially cancel.","pith_inferences":["The distribution-based validation approach should extend to other drifting metrics, such as $T_2^*$ itself or single-gate fidelities, whenever the drift is driven by the same low-frequency noise.","A direct experimental realization on a singlet-triplet qubit with independent noise spectroscopy would test whether the K-S-based error grid reproduces the known model separation in real hardware.","The K-S test ignores temporal correlations in the error-rate time series; a test that uses the power spectrum of the drift could give even stronger model discrimination.","For error attribution, the observed non-additivity suggests that per-mechanism error budgets should be quoted as ensemble statistics, not for individual noise realizations."],"forward_implications":["Instead of reporting a single RB error rate, the distribution of error rates across passes can be reported and used to constrain noise models.","The test's power depends on the total experimental duration relative to the lowest noise frequency; adding delays between passes extends the frequency coverage.","Noise models with high-frequency components (up to $10^7$ Hz in this study) act as strong discriminators against models lacking such components.","Models with only low-frequency noise components have large rejection thresholds and weak discriminating power even with 100 independent noise realizations.","Error attribution on a validated model is not guaranteed to be additive: separate contributions can sum to more or less than the parent trajectory's error."],"supporting_citations":[{"why":"Defines the randomized benchmarking protocol that produces the error rate whose pass-to-pass distribution is the paper's main observable.","marker":"[9]"},{"why":"Establishes that low-frequency noise causes non-exponential fidelity decay in RB, the physical phenomenon that makes the error-rate distribution carry model information.","marker":"[13]"},{"why":"Demonstrates experimentally that drift in quantum information processors can be detected and tracked, motivating the use of drift as a validation signal.","marker":"[15]"},{"why":"Supplies experimental evidence of low-frequency 1/f charge noise in semiconductor quantum dots, grounding the assumed noise family in real devices.","marker":"[27]"},{"why":"Reviews 1/f noise in solid-state quantum information, supporting the physical relevance of the OU-based noise model.","marker":"[32]"},{"why":"Shows that a sum of Lorentzians (equivalently a sum of OU processes) can approximate 1/f noise, justifying the paper's construction of the noise model.","marker":"[56]"},{"why":"Provides the discrete-time power spectral density used to account for aliasing in the wall-clock simulations and justify the 1 GHz sampling rate.","marker":"[58]"}],"fun_headline_variants":["RB drift distribution rules out false qubit noise models","KS test on RB rates debunks qubit noise models","Drift in RB error rates falsifies qubit noise models","Repeated RB passes expose false noise models","Distribution of drifting RB rates rejects wrong models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the true device noise lies within the family of sum-of-OU spectra used in the paper; if the real noise has a different spectral shape or non-Gaussian components that this family cannot represent, the specific discrimination and attribution results need not transfer.","fun_headline_variants_meta":{"raw":{"variants":["RB drift distribution rules out false qubit noise models","KS test on RB rates debunks qubit noise models","Drift in RB error rates falsifies qubit noise models","Repeated RB passes expose false noise models","Distribution of drifting RB rates rejects wrong models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000868,"raw_usage":{"total_tokens":3731,"prompt_tokens":883,"completion_tokens":2848,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":499,"completion_tokens_details":{"reasoning_tokens":2773}},"tokens_in":499,"tokens_out":2848,"duration_ms":19761,"temperature":1.0,"reasoning_tokens":2773,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T10:57:46.963781+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the power spectral density of the charge and magnetic noise of a singlet-triplet qubit from $10^{-3}$ Hz to $10^{7}$ Hz and compare it to the equal-power, one-OU-per-decade spectrum with the stated cutoffs and powers; if the measured spectrum deviates beyond what the OU family can represent, or if a non-Gaussian two-level fluctuator is present, the K-S error-rate grid computed from this family will not describe the real device. Alternatively, run the wall-clock RB protocol on a device with a known non-Gaussian noise source and check whether the K-S test still separates the ten Gaussian models: a failure to reject all models would show the Gaussian spectral-shape assumption is essential.","supporting_citations":[{"cited_title":"Proctor, M","cited_arxiv_id":null,"evidence_quote":"Demonstrates experimentally that drift in quantum information processors can be detected and tracked, motivating the use of drift as a validation signal."},{"cited_title":"Struck, A","cited_arxiv_id":null,"evidence_quote":"Reviews 1/f noise in solid-state quantum information, supporting the physical relevance of the OU-based noise model."},{"cited_title":"Maraj-Zygm¸ at, G","cited_arxiv_id":null,"evidence_quote":"Shows that a sum of Lorentzians (equivalently a sum of OU processes) can approximate 1/f noise, justifying the paper's construction of the noise model."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the discrete-time power spectral density used to account for aliasing in the wall-clock simulations and justify the 1 GHz sampling rate."}],"review_version":1}