{"id":"ea388105-ab7a-4cc4-9cfe-018315c6eb0f","arxiv_id":"2501.10414","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A standard conformal prediction wrapper gives near-nominal coverage for simulated two-qubit measurement distributions in four and twelve dimensions.","lead":"This paper applies classical conformal prediction to probability vectors obtained from simulated two-qubit quantum circuits. It is a quick test of whether distribution-free uncertainty guarantees transfer to quantum-generated data, with reported coverage close to the target level in four and twelve dimensions.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The coverage guarantee is demonstrated only for the empirical shot-frequency vector stored as y, not for the exact Born-rule probability vector; the abstract and Section 4.4 overstate the claim by referring to target probabilities.","rationale":"The reader's weakest assumption identifies exactly the same risk: the paper treats measured frequencies as the target y and evaluates coverage on those same vectors, so the split-conformal guarantee applies to empirical shot-noise vectors rather than exact quantum distributions. I agree with that assessment. The concern is load-bearing because the abstract and Section 4.4 describe coverage of 'target probabilities' deriving from quantum processes, which a careful reader would take to mean the true Born-rule distribution. Split conformal prediction is distribution-free, but not target-free: it guarantees the prescribed coverage for the calibration/test response variable as defined, and here that variable is contaminated by finite-shot noise. The paper does provide public code and a standard conformal pipeline, so the empirical-vector claim is likely sound; the problem is the overreach in the stated conclusion. A statevector-based re-evaluation is a direct, decisive check: if exact-probability coverage also holds near 1−alpha, the central claim survives; if not, the paper should be rescoped. The reader's verdict of CONDITIONAL already captures this, so my stress-test does not change the verdict. I am not raising any objection based on novelty, and I am not disputing the validity of split conformal prediction as applied to empirical vectors; the concern is solely about matching the claim to the evaluated target.","tokens_in":9301,"tokens_out":2790,"duration_ms":31928,"concrete_test":"Re-run the full pipeline with y replaced by the exact Born-rule probabilities for each circuit, computed via Qiskit's statevector simulator (or analytic enumeration) rather than finite-shot frequencies. Use the same features, train/cal/test splits, random-forest regressor, and ℓ2/ℓ∞ conformal scores. If empirical test coverage of exact probability vectors stays at or above 1−alpha in both the 4D and 12D settings, the original wording is justified. If coverage falls below nominal, the paper must be revised to state that guarantees apply to finite-shot empirical vectors only; as a secondary check, vary shots (256, 1024, 4096) and show whether coverage of exact vectors degrades systematically, which would confirm shot noise as the cause.","verdict_should_be":"UNCHANGED","load_bearing_attack":"For the central claim to hold, the conformal set built from calibration residuals must contain the exact quantum probability vector of a test circuit with probability at least 1−alpha. Split conformal prediction guarantees coverage of the response variable actually used in calibration and testing. In Section 2.4 and Listing 1, that response variable is the vector of measured frequencies from 1,024 shots; Section 4 evaluates coverage by checking whether those same empirical vectors fall inside the conformal set. Thus the finite-sample guarantee covers the random empirical vector, not the deterministic exact distribution. Even if the regressor were perfect, a test circuit's empirical frequency vector carries shot noise of magnitude roughly O(1/sqrt(N_shots)), and nothing in the conformal score accounts for the distance between empirical frequencies and exact probabilities. The abstract's phrase 'target probabilities derive from inherently quantum processes' and Section 4.4's claim of coverage in the 'quantum measurement setting' suggest the stronger, exact-probability claim. If the intended target is instead the empirical vector, the claim is correct but must be rescoped; as written, the evidence does not establish coverage of the quantum probability distribution itself. This is the load-bearing gap because it separates the stated contribution from the evaluated object. Exchangeability of circuits is a secondary concern and holds in simulation, so the primary issue is target definition, not the conformal machinery.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript applies split-conformal prediction with a multi-output random forest regressor to probability vectors obtained from random 2-qubit quantum circuits. Two settings are considered: a single-basis Z measurement yielding 4-dimensional output vectors, and a multi-basis Z/X/Y measurement yielding 12-dimensional concatenated vectors. The conformal method is the standard split-conformal quantile threshold on ℓ2 or ℓ∞ residuals, evaluated on held-out test circuits. The paper reports that empirical coverage tracks the nominal 1−α level and that set sizes grow with dimension and circuit complexity, and concludes that classical conformal wrappers can provide coverage guarantees for quantum-generated probability distributions. The code and notebooks are made publicly available.","tokens_in":9551,"tokens_out":4012,"duration_ms":42656,"significance":"If the result is read as coverage of the empirical shot-frequency vectors actually used in calibration and testing, the method is sound: split-conformal prediction is correctly applied, exchangeability holds in the simulated setting, and the empirical coverage check is properly separated from calibration. That said, this is not a new methodological result; it is a standard application of existing multi-output conformal prediction to quantum-circuit data. The stronger reading of the paper—that coverage is guaranteed for the exact Born-rule probability vector of a test circuit—is not established by the experiments, because the response variable is the finite-shot empirical frequency vector. The most valuable parts of the paper are its transparent pipeline and reproducible implementation, together with an honest discussion of exchangeability assumptions; the main limitation is that the central claim is stated more strongly than the evaluated object. After rescoping the claims and adding quantitative results, the paper could be a useful demonstration for practitioners in quantum machine learning.","major_comments":[{"comment":"The coverage guarantee is demonstrated for the empirical shot-frequency vector, not for the exact quantum probability vector. In Sec. 2.4 and Listing 1, y is constructed from measured frequencies (default 1,024 shots per circuit, Sec. 3.1), and Sec. 4 evaluates coverage by checking whether those same empirical vectors fall inside the conformal set. Split-conformal validity (Sec. 2.6, Eq. (2)) applies to the response variable used in calibration and testing; it does not transfer automatically to the exact Born-rule probability vector, which differs from the empirical vector by shot noise of order O(1/sqrt(N_shots)). Nothing in the conformal score accounts for this estimation error. The abstract's phrase 'target probabilities derive from inherently quantum processes' and Sec. 4.4's claim of coverage in the 'quantum measurement setting' therefore overstate what is established. This is the load-bearing gap: the stated contribution is coverage of quantum probability distributions, while the evaluated object is a random empirical vector. I recommend either rescoping the paper's claims to empirical measurement vectors or augmenting the evaluation with statevector-derived exact probabilities as ground truth and a score that incorporates the estimation error.","section":"Sec. 2.4, Listing 1, Sec. 4.4, Abstract"},{"comment":"The experimental results are reported only qualitatively. Phrases such as 'coverage near 90% is typically reached' (Sec. 4.1) and 'a typical coverage table reads something like' (Sec. 4.2) are illustrative rather than measured. The manuscript does not provide a concrete table or figure with the actual coverage, radius, MSE, sample size, or number of test points for each alpha and each setup. Since the central claim is empirical, these numbers need to be reported explicitly, together with standard errors or confidence intervals for the coverage estimates.","section":"Sec. 4.1 and Sec. 4.2"},{"comment":"The sample sizes used in the reported experiments are not tied to the displayed results. Section 3.1 describes toy datasets of '200–500 circuits' and large datasets of '20,000 or 50,000 circuits', but Sec. 4 never states which configuration produced the quoted coverage values. Without this information, the reader cannot judge the stability of the coverage estimates, especially for α = 0.05 where a small test set would give a noisy estimate. Please specify, for every reported result, the dataset size, split sizes, number of shots, and the number of test samples.","section":"Sec. 3.5 and Sec. 4"}],"minor_comments":[{"comment":"There are typos in Listing 1: 'len(cal cal)' should be 'len(cal)', and 'Y cal[i]' is missing a closing bracket in the residual computation.","section":"Listing 1"},{"comment":"The sentence noting that predictions are not constrained to the simplex is useful, but the paper should clarify whether the conformal set is intersected with the probability simplex before coverage is evaluated. If not, the set can contain non-normalized vectors, which weakens the interpretation of coverage for 'probability' outputs.","section":"Sec. 2.4"},{"comment":"The phrase 'distributional conformal prediction' is not standard for the procedure described; the method is ordinary split conformal prediction with a scalar residual. Consider using standard terminology to avoid implying a new variant.","section":"Sec. 2.6"},{"comment":"The duplicate-dropping step based on feature columns may remove samples that are arguably distinct; if two circuits share the same feature vector but have different output distributions, dropping one changes the effective data distribution. Please state how often duplicates were dropped and whether any duplicates remained.","section":"Sec. 3.1"},{"comment":"Reference [5], attributed to Cherubin et al., appears mis-titled relative to the actual paper listed; please verify all reference titles and venue information.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper is essentially a clean application of existing split-conformal prediction to quantum-circuit data; the methodological novelty is low. The main issue is that the abstract and conclusions claim coverage for quantum probability distributions, whereas the experiments only establish coverage for finite-shot empirical frequency vectors. This is fixable by rescoping the claims and by adding a statevector-based evaluation. I would not reject the paper outright, but the revised version must either clearly state that the target is the empirical measurement vector or provide the additional analysis needed to support the stronger claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing you should know: this is a clean, honest application of split conformal prediction to simulated quantum measurement vectors, and the conformal math is right for what it actually evaluates. The problem is that what it evaluates is not what the abstract claims.\n\nThe paper builds a 4D and 12D output from 2-qubit circuits, trains a random forest, calibrates a residual threshold, and checks coverage on held-out circuits. That is textbook split conformal, correctly described in Eqs. (1)-(2). The code is public, and the authors are transparent that exchangeability holds because the circuits are i.i.d. from a simulator. Credit where due: the exposition is clear, the pipeline is reproducible, and there is no hidden fitted constant.\n\nThe soft spot is load-bearing. The response variable y is the empirical frequency vector from 1,024 shots, and coverage is evaluated by checking whether those same empirical vectors fall in the conformal set. Split conformal guarantees coverage of the response variable used in calibration, i.e., the noisy empirical vector, not the exact Born-rule probability. The abstract says coverage holds for \"target probabilities\" from quantum processes, and Section 4.4 says coverage is near-nominal in the \"quantum measurement setting.\" That is overstated. The test does not establish coverage of the true distribution; it establishes coverage of the finite-shot estimate. The difference is O(1/sqrt(N_shots)), and nothing in the score accounts for it.\n\nTwo minor soft spots: the results are reported qualitatively (\"coverage often lands around 95%-97%\") without tables, error bars, or exact numbers for the main experiments; and there is no direct comparison to Park and Simeone's quantum conformal work, which is the obvious baseline. The paper's own equations show the method reduces to standard conformal, so the novelty is the 12D multi-basis pipeline, not the theory.\n\nIf the claims are rescoped to \"empirical shot-frequency vectors,\" the paper is correct and useful as a template. As written, it overclaims. That is a fixable flaw, not a fatal one.\n\nWho benefits: practitioners in quantum ML who want a ready-made conformal wrapper for simulated measurement distributions. It is not a research breakthrough, but it is a legitimate application with a clear writing-up of the standard machinery.\n\nI would send it to peer review, because the target-definition issue is exactly the kind of thing a good referee should catch and push the authors to fix, and the method itself deserves a fair hearing. It needs a major revision before publication.","headline":"Correct but overstated: the conformal wrapper is standard and valid for empirical shot-frequency vectors, but the paper's central claim about covering true quantum probabilities is not supported by the evidence.","tokens_in":10070,"tokens_out":1467,"would_cite":false,"duration_ms":16717,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G15","68T05","81P68"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that classical split-conformal prediction, applied to probability vectors estimated from 2-qubit circuit measurements, yields empirical coverage near 1−α in both 4-dimensional and 12-dimensional output spaces.","keywords":["conformal prediction","quantum machine learning","multi-output regression","distribution-free coverage","multi-basis measurement","uncertainty quantification","random circuit","two-qubit circuit"],"falsifier":"Run the same pipeline but evaluate coverage against the exact noiseless quantum distribution obtained by statevector simulation instead of the finite-shot empirical vector; if coverage against the exact distribution falls substantially below 1−α while empirical-vector coverage stays near nominal, the paper's stated guarantee does not extend to true quantum probabilities.","tokens_in":9099,"feed_emoji":"⚛️","tokens_out":5580,"duration_ms":52049,"temperature":0.7,"pith_summary":"This paper asks whether conformal prediction, a distribution-free recipe for turning any regression model's errors into finite-sample coverage guarantees, still works when the outputs it wraps are probability vectors produced by quantum circuits. To test this, the author generates random 2-qubit circuits, measures each in the computational basis (a 4D outcome vector) or in Z, X, and Y bases (a 12D concatenated vector), trains a multi-output random forest on classical circuit features, and applies distributional conformal calibration with a single residual norm. The reported experiments show empirical test coverage at or above 1−α for α values such as 0.05, 0.10, and 0.30, in both the 4D and 12D settings. The paper reads this as evidence that classical conformal wrappers can supply rigorous uncertainty quantification for quantum-generated predictions, provided calibration and test circuits are exchangeable.","feed_headline":"Conformal sets cover quantum outputs at promised rate","feed_subtitle":"Tests on simulated 2-qubit circuits show near-90% coverage in 4D and 12D measurement spaces.","key_machinery":"The engine is distributional conformal prediction with a single scalar nonconformity score. For calibration samples the score is r_i = ||y_i − f(x_i)||_2 or the ℓ∞ norm; the sorted calibration scores supply the threshold τ_α = r_(⌈(1−α)(n+1)⌉); and the prediction set for a new point is the ball or hypercube C_α(x) = {y : ||y − f(x)|| ≤ τ_α}. Exchangeability of calibration and test circuits is what lets the finite-sample coverage statement hold.","core_discovery":"The central claim is that a classical split-conformal wrapper, using only exchangeable calibration data and a scalar residual norm, delivers finite-sample coverage close to the nominal level 1−α for multi-output probability distributions arising from quantum measurements. Concretely, with 2-qubit random circuits and a multi-output random forest regressor, the 4D Z-basis outputs and the 12D Z/X/Y concatenated outputs both show empirical coverage near nominal, for example around 90 percent at α = 0.10, at the cost of larger residual radii in 12D. The paper concludes that classical conformal methods remain valid when the target probabilities originate in quantum processes, and that imperfect regression models are compensated by enlarging the conformal threshold.","pith_inferences":["The headline result is not that quantum data require new conformal machinery; any exchangeable regression setup would show near-nominal coverage, so the paper demonstrates compatibility rather than a new coverage theorem.","Because the target vectors are shot-count frequencies, the finite-sample guarantee does not cover shot noise: re-measuring the same circuit produces a fluctuating empirical vector that lies outside the conformal construction as stated.","A testable extension is to set the target to the exact statevector probabilities while keeping the empirical vectors as inputs; the drop in coverage would quantify how much of the reported validity is an artifact of finite shots.","On drifting hardware, exchangeability fails, so an online or adaptive recalibration of thresholds would be needed to maintain the guarantee, which the paper itself flags as an open direction."],"forward_implications":["Near-nominal coverage extends to 12D concatenated multi-basis measurement vectors, not just simple 4D single-basis outputs.","Even a deliberately un-tuned random forest regressor can yield valid coverage because the conformal threshold expands to absorb model error.","The same wrapper applies whenever calibration and test circuits are drawn exchangeably, so simulated and, in principle, hardware data fit the same procedure.","Larger output dimension increases the conformal radius and set volume, so coverage comes with a cost that grows with the number of measurement bases."],"supporting_citations":[{"why":"Supplies the quantum conformal prediction precedent and the random-circuit/noise context the experiments build on.","marker":"[1]"},{"why":"Provides the finite-sample coverage theorem that justifies using the sorted calibration quantile as the threshold.","marker":"[6]"},{"why":"Introduces ellipsoidal conformal sets for multi-dimensional outputs, the template for multi-output coverage sets.","marker":"[2]"},{"why":"Provides calibrated multiple-output quantile regression, the multi-output method the random forest pipeline extends.","marker":"[3]"},{"why":"Supplies the basis-rotation measurement constructions for X and Y bases and the quantum-process learning context.","marker":"[9]"}],"fun_headline_variants":["Classical conformal sets cover quantum outputs","Split-conformal wraps quantum multi-outputs to promised coverage","Quantum data, classical coverage: 90% hit rate","Conformal prediction tames 12D quantum measurement spaces","Classical guarantees for quantum probabilities"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the vector of measured frequencies from a finite number of shots (1,024 by default) is the true target y, so the conformal guarantee covers that empirical vector rather than the exact quantum probability distribution from which the shots were drawn.","fun_headline_variants_meta":{"raw":{"variants":["Classical conformal sets cover quantum outputs","Split-conformal wraps quantum multi-outputs to promised coverage","Quantum data, classical coverage: 90% hit rate","Conformal prediction tames 12D quantum measurement spaces","Classical guarantees for quantum probabilities"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000106,"raw_usage":{"total_tokens":999,"prompt_tokens":865,"completion_tokens":134,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":481,"completion_tokens_details":{"reasoning_tokens":61}},"tokens_in":481,"tokens_out":134,"duration_ms":1936,"temperature":1.0,"reasoning_tokens":61,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:39:59.261575+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same pipeline but evaluate coverage against the exact noiseless quantum distribution obtained by statevector simulation instead of the finite-shot empirical vector; if coverage against the exact distribution falls substantially below 1−α while empirical-vector coverage stays near nominal, the paper's stated guarantee does not extend to true quantum probabilities.","supporting_citations":[{"cited_title":"Quantum Conformal Prediction for Reliable Uncertainty Quantification in Quantum Machine Learning,","cited_arxiv_id":null,"evidence_quote":"Supplies the quantum conformal prediction precedent and the random-circuit/noise context the experiments build on."},{"cited_title":"Conformal Prediction for Multi-Dimensional Time Series by Ellipsoidal Sets,","cited_arxiv_id":null,"evidence_quote":"Introduces ellipsoidal conformal sets for multi-dimensional outputs, the template for multi-output coverage sets."},{"cited_title":"Calibrated Multiple-Output Quantile Regression with Representation Learning","cited_arxiv_id":"2110.00816","evidence_quote":"Provides calibrated multiple-output quantile regression, the multi-output method the random forest pipeline extends."},{"cited_title":"Learning to Predict Arbitrary Quantum Processes,","cited_arxiv_id":null,"evidence_quote":"Supplies the basis-rotation measurement constructions for X and Y bases and the quantum-process learning context."}],"review_version":1}