{"id":"29dcad98-bb76-44e0-af15-a5f33ed6e130","arxiv_id":"2412.00078","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"An adaptive Bayesian pipeline using Hamiltonian and sequential Monte Carlo learns IBMQ qubit parameters more accurately than Qiskit's built-in characterization for a fixed number of measurements.","lead":"This thesis tests Bayesian inference with advanced Monte Carlo sampling to learn the parameters of open quantum systems from measurements on IBMQ devices. It reports that these methods outperform Qiskit's built-in characterization at the same measurement budget, offering a calibration alternative for NISQ hardware.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The weakest link is the Markovian, constant-rate channel model used for IBMQ data; if the hardware noise is non-Markovian, the Bayesian posterior targets an effective, sequence-dependent decay rather than a fixed T2, and the claimed equal-budget advantage over Qiskit is not established.","rationale":"The reader's weakest_assumption identifies model misspecification as the main risk; I agree and sharpen it. The specific experiments used—Hahn echo and echoed Ramsey—are designed precisely to suppress low-frequency, quasi-static noise, which means the experiment itself acknowledges non-Markovian behavior while the likelihood model assumes it away. This is the most load-bearing point because the thesis's headline result is an empirical, quantitative comparison: better estimates than Qiskit for the same number of measurements. If the likelihood is misspecified, the posterior mean estimates an effective, protocol-dependent parameter rather than the intended T2, and the comparison with Qiskit no longer measures relative estimator efficiency. Secondary issues such as missing error bars or exact experimental timing are real but could be addressed by additional repetition; model misspecification changes the interpretation of every reported estimate. The thesis is a useful overview, and the advanced Monte Carlo machinery is not itself suspect, so the appropriate verdict remains CONDITIONAL: the empirical claim needs the additional validation described in the concrete test before it can be accepted as stated.","tokens_in":48202,"tokens_out":3822,"duration_ms":42517,"concrete_test":"Run a posterior predictive check on held-out experimental settings from the same IBMQ sessions: after inference from the data used in §5.2, draw parameters from the posterior and simulate outcomes for (i) Hahn echo with a different set of echo spacings, (ii) Ramsey with different free-evolution times, and (iii) a CPMG-like sequence with multiple refocusing pulses. If observed frequencies fall outside the posterior predictive intervals by more than shot noise on at least two of five qubits, the Markovian Kraus likelihood is rejected and the reported advantage cannot be attributed to the estimator.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central empirical claim depends on the likelihood model built in §5.1.2–§5.1.5: a single qubit undergoing coherent precession plus Markovian amplitude damping (rate Γ1=1/T1) and phase damping (rate Γφ), with constant rates throughout the experiment. This likelihood is applied to IBMQ data in §5.2.2–§5.2.5. If the true hardware noise is non-Markovian, as is typical for superconducting qubits, the posterior does not concentrate on a fixed physical T2; it estimates an effective decay rate that depends on echo spacings, pulse order, and noise correlation time. Consequently, the comparison with Qiskit at equal measurement count does not isolate estimator performance: it compares two misspecified exponential fits, and the Bayesian credible intervals are likely overconfident. The text reports no posterior predictive checks, no validation on pulse sequences different from those used for inference, and no repeated runs across qubits or days, so the outperformance claim is not shielded from this failure mode.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This thesis presents a Bayesian approach to learning the parameters of open quantum systems, combining Bayesian experimental design with advanced Monte Carlo methods (HMC/NUTS, SMC with MCMC kernels, and tempered likelihood estimation). The first several chapters provide an extensive tutorial on Bayesian inference and Monte Carlo sampling. The final chapter applies these methods to characterize IBMQ hardware, estimating qubit frequency, T1, and T2 from Hahn echo, energy relaxation, Ramsey, and echoed Ramsey experiments. The central empirical claim is that, for the same number of measurements, the proposed Bayesian protocol outperforms Qiskit's built-in characterization functions. The paper includes code in a GitHub repository and a thorough literature overview.","tokens_in":1291,"tokens_out":1858,"duration_ms":49813,"significance":"If the empirical claim were fully established, the paper would demonstrate a practically relevant advantage of advanced Monte Carlo methods for quantum device characterization, with direct implications for calibration and metrology. The theoretical review is accurate and comprehensive, the synthetic benchmarks in Chapter 4 illustrate the behavior of the algorithms, and the availability of code is a clear strength. The claim itself is falsifiable and of interest to the quantum characterization community. However, the hardware-based comparison with Qiskit is not yet supported by the reported evidence, which limits the significance of the paper in its current form.","major_comments":[{"comment":"The central claim that the Bayesian method 'surpasses the built-in characterization functions of Qiskit for the same number of measurements' is not supported by the evidence presented. The results appear to be reported as point estimates without error bars, repeated-run statistics, or a detailed account of the number of measurements and device drift. Without such statistics, the reported outperformance could be within shot-to-shot or day-to-day variability. A quantitative comparison with uncertainties (e.g., standard errors over repeated experiments) is needed to establish the claim.","section":"Section 5.2.2-5.2.5"},{"comment":"The likelihood model assumes a single qubit undergoing coherent precession plus Markovian amplitude and phase damping with constant rates. If the actual IBMQ noise is non-Markovian or time-dependent, as is typical for superconducting qubits, the posterior targets an effective, sequence-dependent decay rather than a fixed T2, and the credible intervals are likely overconfident. The manuscript does not report posterior predictive checks, validation on pulse sequences different from those used for inference, or repeated runs across qubits or days. At minimum, a posterior predictive check and a stability analysis across data subsets would be needed to shield the outperformance claim from this failure mode.","section":"Section 5.1.2-5.1.5 and Section 5.2.2-5.2.5"},{"comment":"The adaptive scheme introduces several free hyperparameters: the proportionality constant for m_k, the exponential time-spacing constant C, and the prior means and variances. The text does not state how these were chosen for the IBMQ experiments, nor does it provide a sensitivity analysis. If these were tuned on the same data used for the Qiskit comparison, the comparison is not a fair test of the two methods. Reporting the chosen values and a small sensitivity scan (or tuning on a separate calibration dataset) is necessary.","section":"Equations (2.127), (2.129), and Section 5.2"}],"minor_comments":[{"comment":"There are numerous typos and grammatical errors (e.g., 'an monotonous', 'the the two last operations', 'a a single one') that should be corrected.","section":"Throughout"},{"comment":"The experimental setup should state the IBMQ device name, the qubit(s) used, the number of shots per measurement point, and whether data were collected in a single session or across multiple days; these details are essential for assessing the reproducibility of the reported advantage.","section":"Section 5.2.1"},{"comment":"The manuscript does not specify which Qiskit functions were used as baseline (e.g., version of Qiskit and the exact built-in characterization routine), making it difficult to reproduce the comparison.","section":"Section 5.2.2-5.2.5"},{"comment":"The notation 'A =∧ B' for 'A estimates B' is non-standard and could be confused with definitions; consider using a more conventional symbol or wording.","section":"Equation (2.36)"},{"comment":"Several references in the literature overview (e.g., [36], [45], [77]) are cited without explaining their specific relation to the methods compared in Chapter 5; a short sentence for each would help the reader.","section":"Section 1.2"},{"comment":"The text refers to 'crimson curves' but the figure colors are not described in the caption; consider referring to them by line style or position.","section":"Figures 2.19 and 2.20"}],"recommendation":"major_revision","confidential_remarks":"This manuscript is a master's thesis rather than a focused journal article; the refereeing standard for a journal would require the empirical claim to be substantiated with statistics and a clear specification of the experimental protocol. The tutorial chapters could be shortened to make room for the missing experimental details. The paper might be better suited to a venue that accepts review-style contributions. The core Bayesian/Monte Carlo methodology is sound and the code availability is a strength."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know. First, this is a master's thesis, not a journal article, and it reads like one: a thorough, often clear tutorial on Bayesian inference and Monte Carlo, followed by an application to IBMQ characterization. Second, the genuine new bit is the demonstration of HMC/NUTS and SMC-with-MCMC on actual hardware data, with a direct comparison against Qiskit's built-in characterization at the same measurement budget. That comparison is the load-bearing claim.\n\nWhat the paper does well: the exposition of SMC, HMC, NUTS, and Bayesian experimental design is accurate and well-organized. The theory sections are standard but correctly stated. The author does a good job of explaining why Liu-West filters fail on multimodal posteriors and why MCMC kernels inside SMC help. The numerical toy problems in Chapter 4 are sensible. The code is on GitHub, which is more than many papers offer.\n\nWhere it's soft: the headline claim is under-supported. Section 5.2 reports point estimates and credible intervals but gives no error bars over repeated runs, no statistics across multiple qubits or days, and no explicit account of how many measurements went into each comparison. The Qiskit baseline is external, which is good, but we don't know how representative the chosen experiments are. More importantly, the likelihood model is a constant-rate Markovian amplitude- and phase-damping channel. Real IBMQ noise is not Markovian on the timescales of these experiments. That means the posterior is estimating an effective decay rate that likely depends on the pulse sequence and echo spacing. The comparison with Qiskit then becomes two misspecified fits, and the Bayesian credible intervals are probably overconfident. The author does not run posterior predictive checks or validate on pulse sequences different from those used for inference, so this failure mode is unaddressed.\n\nThat said, this concern is a limitation, not a fatal flaw. The central idea — using more robust posterior sampling for quantum device calibration — is sound, and the empirical demonstration is a useful data point. It's an incremental contribution, but it's a real one.\n\nI'd bring this to a reading group only if someone is actively working on quantum characterization. I wouldn't cite it in the next 12 months until the empirical claims are hardened with repeated-run statistics and predictive checks. But I would send it to peer review: a good referee can push the author to add the missing details, and the field benefits from having this comparison on record.","headline":"A well-written thesis that applies standard Bayesian/Monte Carlo machinery to real IBMQ data, but the headline claim about beating Qiskit needs more experimental support before it fully convinces.","tokens_in":48954,"tokens_out":1921,"would_cite":false,"duration_ms":18385,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F15","65C05","81P68","81P50"],"pacs":[],"model":"deepseek-v4-flash","headline":"This thesis claims that Bayesian inference with modern Monte Carlo sampling can characterize open quantum systems on real quantum hardware, and that the resulting estimates beat the device vendor's built-in characterization routines when…","keywords":["Bayesian inference","open quantum systems","quantum characterization","Hamiltonian Monte Carlo","sequential Monte Carlo","Kraus operators","qubit decoherence","quantum device calibration"],"falsifier":"Run the same four experiments on a qubit whose noise is deliberately made non-Markovian, for instance by coupling it to a two-level defect or by letting the T1 and T2 rates drift partway through the data collection, and compare the Bayesian credible intervals with independently measured parameter values. If the credible intervals exclude the independent values while the vendor's built-in routines stay close, the model-misspecification concern is confirmed. A simpler version: interleave the Bayesian characterization with a separate measurement method on the same qubit and check whether the intervals cover the independent result.","tokens_in":47968,"feed_emoji":"⚛️","tokens_out":11409,"duration_ms":101874,"temperature":0.7,"pith_summary":"This thesis argues that Bayesian inference stops being a theoretical ideal and becomes a practical characterization tool once it is paired with the right numerical samplers. The author constructs likelihood models for a single qubit undergoing spin precession, energy relaxation, and dephasing, using a Kraus-operator description of the open-system dynamics, and infers the qubit's frequency, T1 (energy decay time), and T2 (dephasing time) from measurements taken on cloud-accessible superconducting hardware. The central claim is that Hamiltonian Monte Carlo and sequential Monte Carlo with Markov kernels produce posterior estimates that are more accurate than the vendor's built-in characterization functions at the same measurement budget. A sympathetic reader would care because the approach extracts more information per measurement and returns a full posterior distribution, including an honest statement of uncertainty, rather than a single point estimate.","feed_headline":"Bayesian sampler beats built-in qubit calibration at equal shots","feed_subtitle":"Modern Monte Carlo turns scarce measurements into sharper T1, T2, and frequency estimates, uncertainty included.","key_machinery":"The load-bearing object is the Kraus operator-sum representation of the qubit's evolution, a standard way of writing a noisy quantum channel as a sum over jump operators, which converts the physical model into a likelihood function the inference can evaluate. For a single qubit this representation is small enough to write analytically: a precession term exp(-i(ω/2)σ_z t), an amplitude-damping channel built from the T1 rate, and a phase-damping channel built from the T2 rate. The paper couples this likelihood to two families of samplers: Hamiltonian Monte Carlo, including the no-U-turn sampler, which uses gradient information to explore the posterior; and sequential Monte Carlo with Markov kernels, including tempered likelihood estimation, which maintains a particle cloud through a sequence of distributions. The choice of samplers is what lets the posterior be represented without a fixed grid or a Gaussian assumption, and that representation is what carries the claimed gain in accuracy.","core_discovery":"The paper's central discovery is that a Bayesian pipeline built around a Kraus-operator likelihood and sampled with advanced Monte Carlo outperforms the vendor's built-in characterization on real hardware. The likelihood encodes the full dynamics: a Hamiltonian rotation with unknown frequency, an amplitude-damping channel with rate 1/T1, and a phase-damping channel with rate 1/T2. On Hahn echo, T1, Ramsey, and echoed Ramsey experiments, the Bayesian posterior mean estimates of these parameters are claimed to be more accurate than the vendor's own characterization routines for the same number of measurements, while also supplying credible intervals. This is presented as evidence that the bottleneck in Bayesian quantum characterization is representation of the posterior, and that replacing particle-filter or Gaussian approximations with HMC/NUTS and sequential Monte Carlo with Markov kernels removes that bottleneck.","pith_inferences":["A practical consequence the author leaves implicit: the same pipeline could close a calibration loop, using the width of the posterior to decide automatically when a qubit needs to be recalibrated, rather than recalibrating on a fixed schedule.","The likelihood construction generalizes to richer noise models: adding non-Markovian degrees of freedom, such as a coupled defect or a time-dependent decay rate, would let the same inference code test whether the simple memoryless model is actually adequate, via Bayes factors or predictive checks.","Comparing against a second independent characterization method, for example randomized benchmarking for gate errors, would turn the reported single-device comparison into a broader test of whether the Bayesian advantage survives across qubits and error sources."],"forward_implications":["Equal measurement budgets yield better parameter estimates, so qubit calibration and drift tracking can be improved without spending more experimental time on characterization.","The posterior distribution provides calibrated credible intervals, so downstream control decisions can be made with a quantitative sense of how much the parameters are actually known.","Because the update is sequential and sample-based, the same machinery can be used online between measurements, making adaptive experiment design practically usable on real devices.","The likelihood-plus-sampler structure is model-agnostic: the same code path applies to any open-system model whose evolution is written in Kraus operators, not just the three channels tested here."],"supporting_citations":[{"why":"Supplies the basic Bayesian quantum characterization protocol and iterative update structure that the thesis extends to open-system noise models.","marker":"[51]"},{"why":"Establishes Bayesian experimental design for quantum parameter estimation, motivating the adaptive measurement strategy used in the hardware experiments.","marker":"[34]"},{"why":"Represents the SMC strategy with clustered Liu-West filtering that the thesis treats as the baseline to improve on.","marker":"[50]"},{"why":"Provides the sequential Monte Carlo review that defines the particle-filter terminology and SMC framework used for posterior sampling.","marker":"[28]"},{"why":"Introduces tempered likelihood estimation, the SMC variant the thesis uses as the more reliable alternative to sequential importance resampling.","marker":"[86]"},{"why":"Supplies the probabilistic-programming implementation of Hamiltonian Monte Carlo and SMC used to run the inference.","marker":"[95]"},{"why":"Supplies the other major Hamiltonian Monte Carlo implementation referenced as the state of the art the thesis adopts.","marker":"[102]"},{"why":"Foundational reference for Hamiltonian Monte Carlo, giving the sampler's construction and geometric motivation.","marker":"[30]"}],"fun_headline_variants":["Advanced Monte Carlo beats Qiskit's built-in qubit calibration","Bayesian Monte Carlo beats Qiskit calibration with same shots","HMC beats vendor qubit calibration at equal measurement counts","Bayes with HMC outperforms Qiskit's default characterization"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole comparison rests on the assumption that each qubit is exactly described by the small model in Section 5.1: one fixed rotation frequency and two constant decay rates, with nothing else happening during the experiment. If real device noise drifts, depends on which gates are used, or remembers the past, the Bayesian estimates can be wrong even though they look confident.","fun_headline_variants_meta":{"raw":{"variants":["Advanced Monte Carlo beats Qiskit's built-in qubit calibration","Bayesian Monte Carlo beats Qiskit calibration with same shots","HMC beats vendor qubit calibration at equal measurement counts","Bayes with HMC outperforms Qiskit's default characterization"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000715,"raw_usage":{"total_tokens":3134,"prompt_tokens":786,"completion_tokens":2348,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":402,"completion_tokens_details":{"reasoning_tokens":2276}},"tokens_in":402,"tokens_out":2348,"duration_ms":14734,"temperature":1.0,"reasoning_tokens":2276,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T11:46:25.373724+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same four experiments on a qubit whose noise is deliberately made non-Markovian, for instance by coupling it to a two-level defect or by letting the T1 and T2 rates drift partway through the data collection, and compare the Bayesian credible intervals with independently measured parameter values. If the credible intervals exclude the independent values while the vendor's built-in routines stay close, the model-misspecification concern is confirmed. A simpler version: interleave the Bayesian characterization with a separate measurement method on the same qubit and check whether the intervals cover the independent result.","supporting_citations":[{"cited_title":"Robust online hamiltonian learning","cited_arxiv_id":null,"evidence_quote":"Supplies the basic Bayesian quantum characterization protocol and iterative update structure that the thesis extends to open-system noise models."},{"cited_title":"Structured filtering","cited_arxiv_id":null,"evidence_quote":"Represents the SMC strategy with clustered Liu-West filtering that the thesis treats as the baseline to improve on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces tempered likelihood estimation, the SMC variant the thesis uses as the more reliable alternative to sequential importance resampling."},{"cited_title":"Wiecki, and Christopher Fonnesbeck","cited_arxiv_id":null,"evidence_quote":"Supplies the probabilistic-programming implementation of Hamiltonian Monte Carlo and SMC used to run the inference."},{"cited_title":"Stan modeling language users guide and reference manual 2.27, 2021","cited_arxiv_id":null,"evidence_quote":"Supplies the other major Hamiltonian Monte Carlo implementation referenced as the state of the art the thesis adopts."}],"review_version":1}