{"id":"308867d0-50ab-4976-84fe-34db96735132","arxiv_id":"2508.21246","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"A DQN agent with a quantum action-selection circuit generates two-qubit quantum sensor circuits that reach normalized QFI=1, the paper's claimed optimum.","lead":"This paper reports a hybrid AI-quantum agent that uses deep reinforcement learning with a quantum circuit for action selection to design two-qubit sensor circuits. The agent is said to find circuits with maximum normalized Quantum Fisher Information (QFI=1) and fewer gates than several baselines in a noise-free simulation.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"QFI normalization in Eq. 1 is internally inconsistent: the text sets the two-qubit theoretical maximum at QFI=4 yet divides by n=2, so the normalized maximum is 2, not 1; the claimed QFI=1 optimum is therefore not calibrated to the stated metric.","rationale":"The single load-bearing issue is the calibration of the QFI metric. Section III-A defines Eq. 1 with denominator n=2 and simultaneously states that normalized QFI=1 corresponds to raw QFI=4. Those two statements are inconsistent: 4/2=2. Since the reward threshold and all reported QFI values depend on Eq. 1, the claim that HCQA reaches the theoretical maximum and finds an optimal N00N circuit is unsupported. This is more fundamental than the other issues (deterministic-looking action selection, modified GPA comparison, missing code), because those are discrepancies in presentation or fairness, whereas the QFI normalization is the very definition of 'optimal' in the paper. A single analytic check can settle it: compute raw QFI for the episode-3 circuit and for a GHZ/N00N reference; if the raw maximum is 4 and the reported normalized value is 1, the normalization is off by a factor of 2. If the authors intended normalization by n², Eq. 1 and the text need correction; if they intended n, then QFI=1 is not optimal. This reinforces the reader's CONDITIONAL verdict rather than overturning it.","tokens_in":12728,"tokens_out":7583,"duration_ms":79837,"concrete_test":"Recompute the QFI of the episode-3 5-gate circuit and of the reference N00N/GHZ state using the standard definition F=4(⟨ψ|J_z²|ψ⟩−⟨ψ|J_z|ψ⟩²) with J_z=(σ_z1+σ_z2)/2. If the reference max is F=4, then Eq. 1 with n=2 yields a normalized max of 2; the paper's QFI=1 would correspond to F=2, i.e. only the standard quantum limit, not the Heisenberg limit. If instead the authors intended normalization by n²=4, they should report F=4 for the episode-3 state. This single check settles whether 'QFI=1' is a theoretical maximum.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim—HCQA finds an optimal QSC with QFI=1—depends entirely on the assertion in Section III-A that 'A normalized QFI of 1 corresponds to the theoretical maximum (e.g., QFI=4 in our two-qubit system).' But Eq. 1 defines QFI = 4(⟨ψ|Z²|ψ⟩−⟨ψ|Z|ψ⟩²)/n, with n=2. For a pure state and generator Z, the standard QFI is 4Var(Z); for a two-qubit collective spin generator the maximum is 4 (e.g. the GHZ-type state), so Eq. 1 gives 4/2=2 at the maximum—not 1. To obtain a normalized maximum of 1 the denominator would have to be n²=4, not n. The text itself acknowledges the raw maximum is 4, which makes the 'QFI=1 = optimum' identification internally inconsistent. Since the reward threshold T=0.95 and all reported 'QFI=1' results use this normalization, the claim that the 5-gate circuit is an optimal N00N design and that HCQA outperforms other agents is unsupported until the metric is corrected and the actual raw QFI of the generated states is reported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes HCQA, a hybrid classical-quantum agent that combines a Deep Q-Network (DQN) with a quantum action-selection circuit, for the automated synthesis of two-qubit quantum sensor circuits (QSCs). The agent selects gates from {Rx, Ry, S} to maximize the Quantum Fisher Information (QFI) while keeping the gate count small. The QFI, computed via Eq. (1) with a threshold T=0.95 and a maximum of 10 actions per episode, is used as the reward. The authors report that the HCQA achieves an average QFI of 1 over 4000 episodes, that episode 3 produces a 5-gate QSC claimed to be an optimal N00N-state design, and that HCQA outperforms the QRA, GAQA, classical DQN, and GPA baselines. The paper also includes hyperparameters, a pseudocode algorithm, and comparisons in Figs. 8 and 9.","tokens_in":13056,"tokens_out":5572,"duration_ms":58265,"significance":"If the claims were fully supported, this would be a modest but useful proof-of-concept showing that a hybrid DQN with a quantum action-selection circuit can synthesize metrologically useful two-qubit sensor circuits. The manuscript provides reproducible implementation details (network architecture, training parameters, pseudocode) and compares against several baselines, which is commendable. However, the central quantitative claim rests on an internally inconsistent QFI normalization, and the action-selection mechanism is described as probabilistic but implemented deterministically. These issues currently undermine the reported 'QFI=1' optimality results and the claimed advantage of quantum action selection. The work is incremental over the authors' prior QRA/GAQA agents, but the hybrid DQN extension is a reasonable direction that can be repaired with corrected metrics and more rigorous evaluation.","major_comments":[{"comment":"The QFI normalization is internally inconsistent. Eq. (1) defines QFI = 4(<ψ|Z²|ψ> − <ψ|Z|ψ>²)/n with n=2. For a two-qubit pure state the standard QFI is 4Var(Z); the text states that the raw theoretical maximum is QFI=4, but dividing by n=2 gives a normalized maximum of 2, not 1. To obtain a normalized maximum of 1, the denominator would need to be n²=4. As written, the reward threshold T=0.95 and all reported 'QFI=1' results are calibrated to a metric whose maximum is not 1. This directly affects the central claim that the 5-gate circuit is optimal and that HCQA outperforms other agents. Please correct the normalization and report the raw QFI values for the generated states.","section":"Section III-A, Eq. (1)"},{"comment":"The action selection is described as producing 'probabilistic action outcomes' that enhance exploration, but the algorithm explicitly selects 'action = argmax(P(a))' after measuring the quantum circuit. This is a deterministic choice. The claimed advantage over classical DQN's deterministic action selection is therefore not realized in the implemented agent; any exploration comes from the epsilon-greedy schedule in the DQN, not from the quantum measurement. Please either implement true sampling from the measured probability distribution or revise the motivation and claims accordingly.","section":"Section III-B, Algorithm I and Fig. 3"},{"comment":"The comparison with the GPA is not a controlled experiment. The HCQA has already been trained for 4000 episodes (Fig. 8), while the GPA is evaluated for only two episodes on a modified QSC with the S gates removed; the HCQA is also run for only two episodes on this modified task. Two episodes provide no statistical evidence, and the assertion that the maximum QFI is 0.5 in the simplified QSC is given without derivation. Please provide a matched training/evaluation protocol, more episodes, and error bars or statistical significance measures before claiming superiority over the GPA.","section":"Section IV-B, Fig. 9"},{"comment":"The claim that the 5-gate QSC 'represents an optimal design for generating the N00N state' is not supported. The circuit itself is not displayed in the text, and no exhaustive search over gate sequences of length ≤5 or lower bound on gate count for the allowed gate set {Rx, Ry, S} is provided. Without such evidence, 'optimal' is an overstatement. Please show the circuit, verify the generated state overlaps the N00N state, and either provide a brute-force optimality check or soften the claim to 'a compact design found by the agent'.","section":"Section IV-A, Episode 3"}],"minor_comments":[{"comment":"Equation (6) defines r_i^{DQN} as a target Q-value rather than a reward. This is inconsistent with the pseudocode in Algorithm I, where the reward is the QFI and the target is computed separately. Please clarify the notation to avoid confusion.","section":"Section III-A, Eq. (6)"},{"comment":"The text refers to 'Fig. 2 shows the circuit of the quantum action selection technique' but the action selection circuit appears in Fig. 3. Earlier, Fig. 2 is the QSC structure. Please renumber or correct the cross-references.","section":"Section III-B, Figures"},{"comment":"The paper says the HCQA 'ran it for three episodes' for the demonstration, but Fig. 8 reports results over 4000 episodes. Please clarify that the three episodes are an illustrative sample and that Fig. 8 is the full training statistics.","section":"Section IV-A"},{"comment":"In the GPA comparison paragraph, the text says 'The QSC in Fig. 9 is the optimal QSC...' but Fig. 9 is the average QFI plot. It is unclear where this QSC is shown. Please point to the correct figure or include the circuit.","section":"Section IV-B"},{"comment":"There is a typo: 'Rest the state' should be 'Reset the state'. Additionally, the target update in the pseudocode is written as 'target = Q(s,a) + α(r + γ max Q(s',a'))' while the standard update would be a direct assignment; please check that the intended Q-learning update is stated clearly.","section":"Algorithm I"}],"recommendation":"major_revision","confidential_remarks":"The manuscript leans heavily on the authors' prior work (QRA, GAQA, GPA), and the comparison to GPA uses a modified environment that may not be directly comparable. The novelty beyond those works is the injection of a DQN into the action-selection loop, which is standard and not deeply analyzed. The most pressing issue is the QFI normalization error, which is fixable but invalidates the current numbers. I would encourage the editor to request the corrected metric and a re-run of the experiments, plus a more rigorous optimality check, before considering publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper is a small, honest increment on the authors' own prior QRL agents, but the headline result rests on a QFI normalization that the paper itself contradicts. The hybrid architecture is plausible, and the paper is clearly written, but the 'optimal' and 'outperform' claims are not supported until the metric is corrected.\n\nWhat's new: the HCQA combines a classical DQN with a quantum circuit that encodes Q-values into rotation angles and selects actions by measurement. That's a reasonable variation on their QRA/GAQA and on existing QRL ideas. The paper gives concrete hyperparameters, a clear algorithm, and a statevector simulation, which is reproducible in principle. They also honestly state that this is a two-qubit, noise-free proof of concept.\n\nThe problems: Eq. 1 defines QFI = 4 Var(Z)/n with n=2. The text says the theoretical maximum raw QFI is 4 for the two-qubit system, so dividing by n gives a maximum of 2, not 1. The claim that QFI=1 corresponds to the theoretical upper bound is therefore off by a factor of two. This is not cosmetic: the reward threshold T=0.95 and every 'optimal' result is described as QFI=1. If the normalized metric is mis-scaled, the agent is actually converging to roughly half the true maximum, and the 'optimal N00N design' claim is unsupported. The authors need to report the raw QFI or correct the normalization.\n\nAlso, the quantum action selection is presented as probabilistic but Algorithm I uses argmax over measured probabilities, making it deterministic. That may be fine, but it contradicts the stated motivation.\n\nThe comparisons are not strong: the GPA comparison removes S gates and changes the action set, so the QFI maximum drops to 0.5; comparing agents on different action sets and different reward scales isn't meaningful. No code or error bars are provided, so I can't tell how stable the 'average QFI of 1' is over seeds.\n\nWho this is for: people working on quantum reinforcement learning for circuit synthesis might read it as an incremental proof of concept. It is not a breakthrough and the toy problem limits its significance.\n\nRecommendation for peer review: it deserves a careful referee rather than a desk reject, because the core idea is coherent and the flaws are fixable. But I would not accept it as is; the authors need to correct the normalization, report raw QFI, and make the comparisons fair.","headline":"A sincere but flawed proof-of-concept: the hybrid agent idea is fine, but the QFI normalization bug undermines the central 'optimality' claim.","tokens_in":13574,"tokens_out":4290,"would_cite":false,"duration_ms":41159,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper reports that a hybrid DQN-plus-quantum-action-selection agent can autonomously learn a five-gate quantum sensor circuit whose normalized Quantum Fisher Information reaches 1, the paper's stated theoretical ceiling for a two-qubit","keywords":["HCQA","DQN","QRL","QSC","QFI","Quantum Action Selection","N00N state","quantum metrology"],"falsifier":"Run the five-gate circuit in a statevector simulator, compute the raw QFI via 4(⟨ψ|Z²|ψ⟩ − ⟨ψ|Z|ψ⟩²) with the same generator Z used in Eq. 1, and check whether the maximum over all gate sequences is 4. If the raw maximum differs, the normalized target of 1 is miscalibrated and the optimality claim does not follow. A second check: compare the generated state's QFI with the QFI of an ideal N00N state under the same generator.","tokens_in":12586,"feed_emoji":"⚛️","tokens_out":12440,"duration_ms":117587,"temperature":0.7,"pith_summary":"The paper proposes HCQA, a reinforcement-learning agent that pairs a classical Deep Q-Network with a small quantum circuit that selects actions probabilistically. HCQA's goal is to generate quantum sensor circuits (QSCs)—sequences of Rx, Ry, and S gates on two qubits—that maximize Quantum Fisher Information, the standard measure of how precisely a state can estimate a phase. The paper's claim is that this hybrid agent reaches a normalized QFI of 1, which it treats as the theoretical ceiling, and that the five-gate circuit it finds in the third episode is the optimal design for producing a N00N state. If true, the result matters because it replaces manual or brute-force circuit design with an agent that discovers metrologically optimal circuits while keeping gate count low.","feed_headline":"Hybrid agent designs 5-gate quantum circuit at sensitivity limit","feed_subtitle":"Classical learning plus quantum action-selection finds the shortest sensor circuit at the two-qubit ceiling.","key_machinery":"The load-bearing mechanism is the QFI-as-reward QSC environment coupled to a quantum action-selection circuit. The quantum state is compressed into a discretized Husimi-Q representation and fed to a DQN; the DQN's output Q-values are normalized via θ = (highest Q(a) / Σ Q(a_i)) · π, encoded into Ry gates, and followed by two Hadamard gates. Measuring the two qubits then yields a probability distribution over the three actions Rx, Ry, and S, and the agent selects the most probable action. QFI, computed as 4(⟨ψ|Z²|ψ⟩ − ⟨ψ|Z|ψ⟩²)/n for n=2, is both the reward signal and the optimization target, so maximizing sensor sensitivity and minimizing circuit complexity become the same learning objective","core_discovery":"The paper's central claim is that a hybrid agent—a classical DQN for policy learning plus a two-qubit quantum circuit that converts Q-values into gate-choice probabilities—can autonomously synthesize an optimal two-qubit quantum sensor circuit. The agent's rewards come from computing QFI after each applied gate; the target is a normalized QFI of 1, which the paper equates with the theoretical maximum (raw QFI 4 for two qubits). In the reported experiments, the agent reaches that ceiling with circuits of seven, six, and finally five gates, and the five-gate circuit is identified as the optimal N00N-state generator. Across 4000 episodes the HCQA's average QFI is 1, above the previous QRA agent","pith_inferences":["A direct test on hardware is feasible: implement the five-gate circuit on a two-qubit device, estimate QFI from measurement statistics, and compare with the statevector value; because the circuit is short, a shortfall would point to calibration or noise rather than to the learning algorithm.","The paper's optimality criterion is gate count at fixed QFI; the six- and seven-gate circuits also reach QFI 1, so 'optimal' should be read as 'shortest found circuit within the chosen gate alphabet', not as a uniqueness proof.","Applying the same hybrid agent to three or more qubits would test whether the normalized QFI ceiling of 1 remains the right target and whether the five-gate solution is a special property of the two-qubit, three-action search space."],"forward_implications":["If the result holds, optimal two-qubit Ramsey-style sensor circuits can be discovered by reinforcement learning rather than hand-designed, and the discovered circuit is short enough to implement on current hardware.","The five-gate solution shows that maximizing QFI and minimizing gate count are compatible objectives in this setting, so the agent is not trading sensing precision for simplicity.","The quantum action-selection circuit provides a built-in probabilistic exploration mechanism, addressing a known weakness of deterministic DQN action selection.","The same QFI-reward formulation can be lifted to other gate alphabets and qubit counts, giving a path toward automated design of larger quantum sensors."],"supporting_citations":[{"why":"Defines the QSC environment and the QFI-maximization task that HCQA extends; also serves as the primary baseline.","marker":"[5]"},{"why":"Provides the GAQA quantum-agent baseline that HCQA is compared against in the evaluation.","marker":"[7]"},{"why":"Introduces the QSC formulation (generalized Ramsey measurement over elementary gates) that the environment implements.","marker":"[16, 17, 18]"},{"why":"Establishes Quantum Fisher Information as the measure of sensing precision that supplies the reward.","marker":"[21, 22, 23, 24]"},{"why":"Supplies the DQN architecture, target network, replay buffer, and Q-learning update used by the classical side of HCQA.","marker":"[25]"},{"why":"Identifies N00N states as the QFI-maximizing target that the five-gate circuit claims to realize.","marker":"[26]"},{"why":"Supplies the discretized Husimi-Q representation used to compress the quantum state into the DQN input.","marker":"[29, 30]"}],"fun_headline_variants":["Hybrid AI designs 5-gate quantum sensor at QFI limit","Classical-quantum agent finds shortest optimal sensor circuit","DQN plus quantum action-selection hits max sensitivity with 5 gates","Agent learns to build 5-gate N00N state for quantum sensing","Optimal two-qubit sensor circuit found by hybrid learning agent"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that a normalized QFI of 1 is the correct theoretical maximum for the two-qubit system (raw QFI 4); if this normalization is miscalibrated, the target and the optimality claim lose their ground.","fun_headline_variants_meta":{"raw":{"variants":["Hybrid AI designs 5-gate quantum sensor at QFI limit","Classical-quantum agent finds shortest optimal sensor circuit","DQN plus quantum action-selection hits max sensitivity with 5 gates","Agent learns to build 5-gate N00N state for quantum sensing","Optimal two-qubit sensor circuit found by hybrid learning agent"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000199,"raw_usage":{"total_tokens":1208,"prompt_tokens":741,"completion_tokens":467,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":485,"completion_tokens_details":{"reasoning_tokens":375}},"tokens_in":485,"tokens_out":467,"duration_ms":5173,"temperature":1.0,"reasoning_tokens":375,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T14:27:18.869379+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the five-gate circuit in a statevector simulator, compute the raw QFI via 4(⟨ψ|Z²|ψ⟩ − ⟨ψ|Z|ψ⟩²) with the same generator Z used in Eq. 1, and check whether the maximum over all gate sequences is 4. If the raw maximum differs, the normalized target of 1 is miscalibrated and the optimality claim does not follow. A second check: compare the generated state's QFI with the QFI of an ideal N00N state under the same generator.","supporting_citations":[{"cited_title":"ReLAQA: Reinforcement Learning- Based Autonomous Quantum Agent for Quantum Applications,","cited_arxiv_id":null,"evidence_quote":"Provides the GAQA quantum-agent baseline that HCQA is compared against in the evaluation."},{"cited_title":"Reinforcement-learning-based matter-wave interferometer in a shaken optical lattice,","cited_arxiv_id":null,"evidence_quote":"Supplies the DQN architecture, target network, replay buffer, and Q-learning update used by the classical side of HCQA."},{"cited_title":"A Quantum Rosetta Stone for Interferometry,","cited_arxiv_id":null,"evidence_quote":"Identifies N00N states as the QFI-maximizing target that the five-gate circuit claims to realize."}],"review_version":1}