{"id":"27400d03-3918-496a-b509-eee822c0b67b","arxiv_id":"2509.07614","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A two-armed bandit's reward probabilities are learned as quantum-circuit rotations on IBM hardware and then used in quantum policy evaluation on an IonQ trapped-ion machine, with noisy but partly correct results.","lead":"The paper's authors teach a two-qubit quantum circuit to reproduce the reward odds of a slot machine from ordinary data, then run a quantum algorithm on an IonQ computer to estimate how good two policies are. The run works only for the simplest circuits; noise pushes the harder versions off target, so this is a feasibility check, not a demonstration of quantum advantage.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section IV never reports the QPE circuit's theta angles; if exact 70%/20% probabilities were used instead of the learned (1.96, 0.91), the QML-to-QPE integration claim is not demonstrated.","rationale":"The reader's weakest assumption identifies exactly the same concern: the QPE circuits in Section IV may have used exact probabilities rather than the learned theta values, and the paper provides no theta values for the QPE run. This is the most load-bearing issue because the paper's headline contribution is the integration of QML and QPE. The concern is addressable with a small artifact or clarification, so it does not warrant rejection; the appropriate verdict remains CONDITIONAL, which is what the reader already assigned. I considered whether the 'quantum advantage' framing in the conclusion is overreach, but that is a secondary interpretive issue: the paper's actual experiments only show feasibility on a two-armed bandit, and the abstract already tempers the claim with 'promising potential.' The missing theta linkage is more central and more concrete. I also considered the lack of classical baselines, but that is a standard limitation rather than a single point where the argument could silently break. The QPE runs themselves are plausible and honestly reported, but without the theta values, the central pipeline cannot be audited. The proposed concrete test directly resolves the ambiguity by comparing histograms generated from learned versus exact parameters.","tokens_in":7314,"tokens_out":2260,"duration_ms":23917,"concrete_test":"Request the exact theta values and circuit-generation code used for Figure 5, or rerun the QPE experiments with the learned parameters from Table I (θ←=1.96, θ→=0.91) and with the exact target parameters (θ←=2 arcsin(√0.7), θ→=2 arcsin(√0.2)). Compare the resulting n=3 and n=4 histograms to Figure 5. If the published histograms match the learned-parameter run but not the exact-parameter run, the integration claim is supported. If they match the exact-parameter run instead, the QML-to-QPE link is unsupported. If they match neither, noise dominates and the reported QPE results are not reproducible from the stated parameters. A quantitative comparison using the same 300-shot count and a distance measure (e.g., total variation distance) would settle which case holds.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is the end-to-end integration: QML learns environment parameters from classical data (Section III), and those learned parameters are then used in QPE on hardware (Section IV). The weakest link is that Section IV.B says the environment is 'as learned in Section III' but never reports the theta values actually inserted into the QPE circuit. Table I gives learned values θ←=1.96, θ→=0.91 for the 70%/20% environment, and the exact values from Eqs. (8)-(9) are θ←=2 arcsin(√0.7)≈1.982, θ→=2 arcsin(√0.2)≈0.927. These are close, so the resulting QPE histograms would be nearly indistinguishable, making the omission hard to detect by eye. If the authors used the exact theoretical angles rather than the learned ones, Figure 5 validates QPE under hardware noise but does not validate the QML learning step. Because the learned and exact angles are so close, the claim is not false on its face, but it is unauditable: no artifact, code, or reported parameters allow a reviewer to confirm the end-to-end pipeline. This is the load-bearing assumption because the paper's stated novelty is 'learned quantum environment is then applied in QPE'; if that connection is not established, the work reduces to two separate hardware demonstrations.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports an end-to-end integration of quantum machine learning (QML) and quantum policy evaluation (QPE) on real quantum hardware. In Section III, the authors learn the rotation parameters of a two-armed bandit environment from classical batch data using gradient-free COBYLA optimization with Q-CTRL noise mitigation on IBM hardware. In Section IV, they implement QPE on IonQ's Forte-1 device, claiming the environment is 'as learned in Section III,' and estimate the values of two policies. The results show learned parameters close to the theoretical inverse-sine formulas (Eqs. 8-9), and QPE histograms that peak near the true values for n=3 but degrade for deeper circuits under noise. The paper concludes that, despite hardware limitations, the QML-QPE integration shows promise for quantum advantage in reinforcement learning.","tokens_in":7694,"tokens_out":5357,"duration_ms":63144,"significance":"If the end-to-end link is fully supported, this is a useful first hardware demonstration of QPE and of using QML to construct quantum environment models from classical data. The paper's strengths include the use of actual quantum hardware for both stages, the choice of a gradient-free optimizer appropriate for noisy devices, the explicit formulas in Eqs. (8)-(9) that allow direct cross-checking of learned parameters, and an honest discussion of noise-induced degradation. However, the central integration claim is currently unauditable because the QPE circuit parameters are not reported, no error bars accompany the hardware histograms, and the loss values in Section III appear inconsistent with the stated MSE cost. The theoretical quantum-advantage claim is inherited from prior work [6]; the present experiments do not themselves compare against classical baselines, so the title's 'quantum advantage' should be read prospectively rather than as a demonstrated result.","major_comments":[{"comment":"The QPE circuits are said to use the environment 'as learned in Section III,' but the exact rotation angles inserted into the circuits are never reported. Table I gives learned values θ←=1.96, θ→=0.91 for the 70%/20% environment, while the exact theoretical values from Eqs. (8)-(9) are θ←≈1.982, θ→≈0.927. These are close enough that the histograms in Fig. 5 would be nearly indistinguishable, so the figures cannot confirm that the learned parameters were actually used. The abstract's claim that 'the learned quantum environment is then applied in QPE' is load-bearing; without reporting the circuit angles (or providing code/data), the end-to-end QML-to-QPE link cannot be audited. Please state the exact θ values used for each QPE run and, ideally, provide the circuit construction details as supplementary material.","section":"Section IV.B / Table I"},{"comment":"The cost function is defined in Section III.C as the MSE between measured and empirical relative frequencies, which for two probabilities in [0,1] can be at most 2. Yet Section III.F reports L_opt ≈ 111.3 for the 70%/20% case and L_opt ≈ 69.3 for the 0%/50% case. These values are incompatible with the stated MSE. Please define the actual loss function (e.g., count-based, weighted, or rescaled MSE), and report the empirical batch frequencies f_data used for each run. Table I's 'Empirical' columns (1.98 and 0.93) coincide exactly with the inverse-sine of the true probabilities 0.7 and 0.2; please clarify whether these are finite-sample estimates or the exact theoretical values, and if finite, provide the sample sizes and measured frequencies.","section":"Section III.C and III.F"},{"comment":"The QPE results are presented as count histograms over 300 runs, but no error bars or confidence intervals are given. Given that each bin count is a binomial random variable, standard errors are easy to compute and would allow the reader to judge whether the differences between n=3 and n=4, and between simulator and hardware, are statistically meaningful. Without these, the qualitative comparison in Section IV.C (e.g., 'more shots are farther away from the true value') is not quantitatively supported. Please add error bars or per-bin counts with uncertainties.","section":"Section IV.C / Figure 5"}],"minor_comments":[{"comment":"The abbreviation 'QPE' is used both for 'quantum policy evaluation' and 'quantum phase estimation' in Section IV.A. Please disambiguate to avoid confusion.","section":"Section II / IV.A"},{"comment":"The text says 'the deviation is 0.015 for both parameters' but does not specify whether this is the absolute difference from the theoretical values in Eqs. (8)-(9) or from the empirical values in Table I. Please clarify.","section":"Section III.F"},{"comment":"Minor typos: '300times' should be '300 times', and 'noticable' should be 'noticeable' in Section IV.C.","section":"Section IV.B"},{"comment":"The hardware name is given as 'forte-1' but elsewhere as 'Forte-1' (and the reference uses 'Forte'). Please standardize the capitalization.","section":"Section IV.B"},{"comment":"The phrase 'quantum advantage' in the title and abstract is stronger than what the experiments demonstrate. The conclusion appropriately says 'potential quantum advantage,' so please align the abstract/title with that more cautious framing, or add a classical baseline comparison.","section":"Conclusion / Abstract"}],"recommendation":"major_revision","confidential_remarks":"The missing theta values in Section IV are the key obstacle: the paper's novelty is the end-to-end QML-to-QPE pipeline, and the current manuscript does not allow a reviewer to verify that the learned parameters, rather than the exact theoretical angles, were used. This is fixable by reporting the angles and ideally providing code/data. I would also encourage the editor to ask for a data availability statement, as no artifacts are referenced. The paper fits the journal's scope as a hardware demonstration, but the 'quantum advantage' framing should be moderated unless a classical resource comparison is added."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"What you should know: this is the first time quantum policy evaluation has been run on real quantum hardware. The conceptual pieces are borrowed—the QPE algorithm from Wiedemann et al., the VQC fitting from standard QML—but the integration is new. If you care about whether these algorithms survive NISQ noise, this is a useful data point.\n\nThe QML half is clean. They learn the two-armed bandit parameters on IBM hardware with COBYLA and Q-CTRL noise mitigation, and the fitted parameters match the closed-form expressions within 0.015. That part holds up. The QPE half also works for the simplest circuit (n=3) on IonQ's forte-1; with n=4 the depth kills it, which is exactly what you'd expect. The comparison across ideal simulator, noisy simulator, and real hardware is honest, and the failure mode is plain.\n\nThe soft spots, in order. First, the central integration claim—that QPE uses the parameters learned in Section III—is not auditable. Section IV.B says the environment is 'as learned in Section III' but never reports the theta values actually inserted into the QPE circuit. The exact theoretical angles for the 70%/20% environment are 1.982 and 0.927; the learned ones from Table I are 1.96 and 0.91. The resulting QPE histograms would look nearly identical either way, so you can't tell from the figures which angles were used. If the authors used the theoretical angles instead of the learned ones, then Figure 5 validates QPE under noise but not the QML learning step. The stress-test note is right to flag this; it's a genuine gap, not an artifact. Second, there's no classical baseline and no attempt to measure any speedup, so the 'quantum advantage' language in the abstract is a hope, not a result. The conclusion is more careful—it admits only the most basic circuit produced expected outcomes—so this is a framing issue, not a claim of demonstrated advantage. Third, the benchmark is a single-state two-armed bandit; the learning task is trivial, and the challenge is purely hardware noise.\n\nWho should read it: people working on quantum RL on NISQ hardware, and anyone tracking where amplitude estimation breaks under decoherence. It deserves a serious referee, and the referee should push for the exact QPE angles and ideally code or run data. The citation pattern is fine; the self-citation to [6] is necessary.\n\nRecommendation: engage with it, but ask the authors to disclose the QPE circuit parameters. That single addition would turn the central claim from plausible to verifiable.","headline":"First hardware run of QPE is a genuine engineering milestone, but the QML-to-QPE integration claim is unauditable because Section IV never reports the actual theta angles used in the QPE circuit.","tokens_in":8124,"tokens_out":2501,"would_cite":false,"duration_ms":25728,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a complete quantum reinforcement-learning pipeline—learning environment parameters from classical offline data with gradient-free QML, then using those parameters in quantum policy evaluation—can run on today's quantu","keywords":["quantum reinforcement learning","quantum policy evaluation","quantum machine learning","two-armed bandit","variational quantum circuits","gradient-free optimization","noise mitigation","quantum hardware"],"falsifier":"Compare QPE output distributions for the same environment and hardware using (a) the learned angles from Table I and (b) the ideal angles derived from the true 70%/20% winning probabilities; if the two distributions are statistically indistinguishable, the claimed end-to-end learning pipeline is not evidenced by the QPE results.","tokens_in":7286,"feed_emoji":"⚛️","tokens_out":10516,"duration_ms":94484,"temperature":0.7,"pith_summary":"The paper tries to show that quantum reinforcement learning can run end to end on current quantum hardware: environment parameters are learned from a batch of classical observations by a gradient-free quantum machine learning routine, and the learned quantum environment is then fed into quantum policy evaluation (QPE) to estimate policy values. The benchmark is a two-armed bandit whose reward probabilities are encoded in two rotation angles of a small quantum circuit. On one commercial quantum platform the angles are trained to within about 0.015 of the ideal values; on a second, trapped-ion platform, QPE with the smallest circuit depth returns distributions peaked near the true policy values, while deeper circuits are degraded by hardware noise. The authors take this as evidence that the QML-plus-QPE pipeline is feasible on today's noisy devices for low-complexity problems, and as a step toward the quadratic sample-efficiency advantage that QPE has over classical Monte Carlo.","feed_headline":"Quantum policy evaluation runs on real hardware from classical data","feed_subtitle":"A two-armed bandit environment is learned on one quantum platform, then used to evaluate policies on a trapped-ion device.","key_machinery":"The carrier of the argument is a small parameterized two-qubit circuit for a two-armed bandit. The action qubit, prepared in |0⟩ or |1⟩, selects an arm; a controlled Ry rotation on the reward qubit, with angle θ← or θ→, sets that arm's reward probability through P(reward)=sin²(θ/2). Learning reduces to finding the two angles that make the circuit's measured reward frequencies match the classical data. The second piece is quantum policy evaluation (QPE): a quantum phase estimation routine that, given a unitary encoding of the policy and environment, returns a quantized estimate of the policy value using O(1/ϵ) quantum samples—quadratically fewer than classical Monte Carlo. The paper's contrib","core_discovery":"The central discovery is a working two-stage pipeline. Classical bandit data (empirical left/right winning frequencies) train the rotation angles θ← and θ→ of a two-qubit environment circuit, using a gradient-free optimizer with a noise-mitigation wrapper; on real hardware the angles end up within 0.015 of the theoretical ideal for true reward probabilities 70%/20%. The learned environment then goes into QPE, run on a trapped-ion quantum computer: with n=3 phase-estimation qubits, estimated policy values concentrate near the true value v=0.45 for a randomized policy; n=4 is visibly corrupted by noise. A deterministic policy is almost unaffected because the environment collapses to one rotati","pith_inferences":["A natural next test is to report the exact learned angles used in the QPE circuits and to compare against circuits built from the ideal angles; that would separate the QML learning quality from the QPE hardware behavior, and it is not auditable from the paper as written.","The same two-stage recipe could be tried on multi-state MDPs, but circuit depth grows quickly with state and action count, so hardware noise—not the learning rule—will likely set the practical limit.","The paper's framing suggests that once error mitigation improves, the quadratic qsample advantage could be demonstrated by increasing n and showing the QPE estimate converging to the true value faster than classical Monte Carlo on equivalent samples; that experiment would operationalize 'quantum advantage' for this benchmark.","If the learned and ideal angles produce indistinguishable QPE outputs, then the current results would be evidence of hardware noise behavior rather than of learning; a reader should treat the end-to-end claim as conditional on that check."],"forward_implications":["If the pipeline holds up, offline classical RL data can be used to instantiate a quantum environment on hardware, removing the need for hand-crafted environment circuits in QPE.","QPE's quadratic sample-complexity advantage becomes practically accessible only when hardware noise is controlled enough to run the deeper n=4 and higher circuits; currently only the shallowest circuit gives reliable estimates.","The noise robustness of deterministic policies suggests that QPE on current hardware will be most reliable in settings where the policy collapses the environment dynamics to a small number of gates.","The small training deviation (around 0.015) indicates that gradient-free QML with error suppression can learn environment parameters well enough for simple benchmarks, so the learning stage is not the current bottleneck."],"supporting_citations":[{"why":"Defines the quantum policy evaluation algorithm and the gate-based two-armed bandit environment circuit that this paper learns and then executes; the source of the QPE sample-complexity claim.","marker":"[6]"},{"why":"Cited to support the choice of gradient-free optimization for noisy quantum hardware.","marker":"[19]"},{"why":"Benchmarks optimizers for variational quantum algorithms; supports the use of gradient-free methods for the QML training.","marker":"[20]"},{"why":"Provides the gradient-free optimizer used to minimize the mean squared error between measured and classical reward frequencies.","marker":"[21]"},{"why":"Supplies the automated deterministic error-suppression workflow integrated into the optimization loop to reduce hardware noise.","marker":"[22]"},{"why":"Benchmarks the trapped-ion quantum computer used for the QPE experiments, providing the hardware context for the noise results.","marker":"[23]"}],"fun_headline_variants":["Quantum policy evaluation trained on classical data runs on hardware","Classical data trains quantum environment; QPE runs on trapped-ion","Two-stage quantum RL: learn environment from data, then evaluate on hardware","Quantum policy evaluation: classical data in, hardware out"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that the QPE circuits run with the rotation angles the QML step learned from the batch data, not with the known true reward probabilities inserted directly; the paper states the environment is 'as learned in Section III' but does not report those angles for the QPE circuit, so that link cannot be checked.","fun_headline_variants_meta":{"raw":{"variants":["Quantum policy evaluation trained on classical data runs on hardware","Classical data trains quantum environment; QPE runs on trapped-ion","Two-stage quantum RL: learn environment from data, then evaluate on hardware","Quantum policy evaluation: classical data in, hardware out"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000703,"raw_usage":{"total_tokens":2977,"prompt_tokens":680,"completion_tokens":2297,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":424,"completion_tokens_details":{"reasoning_tokens":2227}},"tokens_in":424,"tokens_out":2297,"duration_ms":22329,"temperature":1.0,"reasoning_tokens":2227,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T21:55:51.669532+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare QPE output distributions for the same environment and hardware using (a) the learned angles from Table I and (b) the ideal angles derived from the true 70%/20% winning probabilities; if the two distributions are statistically indistinguishable, the claimed end-to-end learning pipeline is not evidenced by the QPE results.","supporting_citations":[{"cited_title":"Quantum policy iteration via am- plitude estimation and Grover search – towards quantum advantage for reinforcement learning,","cited_arxiv_id":null,"evidence_quote":"Defines the quantum policy evaluation algorithm and the gate-based two-armed bandit environment circuit that this paper learns and then executes; the source of the QPE sample-complexity claim."},{"cited_title":"Quantum circuit evolution on NISQ devices,","cited_arxiv_id":null,"evidence_quote":"Cited to support the choice of gradient-free optimization for noisy quantum hardware."},{"cited_title":"Benchmark- ing of different optimizers in the variational quantum algorithms for applications in quantum chemistry,","cited_arxiv_id":null,"evidence_quote":"Benchmarks optimizers for variational quantum algorithms; supports the use of gradient-free methods for the QML training."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the gradient-free optimizer used to minimize the mean squared error between measured and classical reward frequencies."},{"cited_title":"Experimental benchmarking of an automated deterministic error-suppression workflow for quantum algorithms,","cited_arxiv_id":null,"evidence_quote":"Supplies the automated deterministic error-suppression workflow integrated into the optimization loop to reduce hardware noise."},{"cited_title":"Benchmarking a trapped-ion quantum computer with 30 qubits,","cited_arxiv_id":null,"evidence_quote":"Benchmarks the trapped-ion quantum computer used for the QPE experiments, providing the hardware context for the noise results."}],"review_version":1}