REVIEW 4 major objections 5 minor 32 references
HCQA: Hybrid Classical-Quantum Agent for Generating Optimal Quantum Sensor Circuits
T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash
Pith's one-line read The paper reports that a hybrid DQN-plus-quantum-action-selection agent can autonomously learn a five-gate quantum sensor circuit whose normalized Quantum Fisher Information reaches 1, the paper's stated theoretical ceiling for a two-qubit
desk verdict A sincere but flawed proof-of-concept: the hybrid agent idea is fine, but the QFI normalization bug undermines the central 'optimality' claim. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the QFI-as-reward QSC environment coupled to a quantum action-selection circuit. The quantum state is compressed into a discretized Husimi-Q representation and fed to a DQN; the DQN's output Q-values are normalized via θ = (highest Q(a) / Σ Q(a_i)) · π, encoded into Ry gates, and followed by two Hadamard gates. Measuring the two qubits then yields a probability distribution over the three actions Rx, Ry, and S, and the agent selects the most probable action. QFI, computed as 4(⟨ψ|Z²|ψ⟩ − ⟨ψ|Z|ψ⟩²)/n for n=2, is both the reward signal and the optimization target, so maximizing sensor sensitivity and minimizing circuit complexity become the same learning objective
What would settle it
Run the five-gate circuit in a statevector simulator, compute the raw QFI via 4(⟨ψ|Z²|ψ⟩ − ⟨ψ|Z|ψ⟩²) with the same generator Z used in Eq. 1, and check whether the maximum over all gate sequences is 4. If the raw maximum differs, the normalized target of 1 is miscalibrated and the optimality claim does not follow. A second check: compare the generated state's QFI with the QFI of an ideal N00N state under the same generator.
Extended reading notes
Core claim
The paper's central claim is that a hybrid agent—a classical DQN for policy learning plus a two-qubit quantum circuit that converts Q-values into gate-choice probabilities—can autonomously synthesize an optimal two-qubit quantum sensor circuit. The agent's rewards come from computing QFI after each applied gate; the target is a normalized QFI of 1, which the paper equates with the theoretical maximum (raw QFI 4 for two qubits). In the reported experiments, the agent reaches that ceiling with circuits of seven, six, and finally five gates, and the five-gate circuit is identified as the optimal N00N-state generator. Across 4000 episodes the HCQA's average QFI is 1, above the previous QRA agent
Load-bearing premise
The load-bearing premise is that a normalized QFI of 1 is the correct theoretical maximum for the two-qubit system (raw QFI 4); if this normalization is miscalibrated, the target and the optimality claim lose their ground.
Editorial extensions
If this is right
- If the result holds, optimal two-qubit Ramsey-style sensor circuits can be discovered by reinforcement learning rather than hand-designed, and the discovered circuit is short enough to implement on current hardware.
- The five-gate solution shows that maximizing QFI and minimizing gate count are compatible objectives in this setting, so the agent is not trading sensing precision for simplicity.
- The quantum action-selection circuit provides a built-in probabilistic exploration mechanism, addressing a known weakness of deterministic DQN action selection.
- The same QFI-reward formulation can be lifted to other gate alphabets and qubit counts, giving a path toward automated design of larger quantum sensors.
Reading between the lines
- A direct test on hardware is feasible: implement the five-gate circuit on a two-qubit device, estimate QFI from measurement statistics, and compare with the statevector value; because the circuit is short, a shortfall would point to calibration or noise rather than to the learning algorithm.
- The paper's optimality criterion is gate count at fixed QFI; the six- and seven-gate circuits also reach QFI 1, so 'optimal' should be read as 'shortest found circuit within the chosen gate alphabet', not as a uniqueness proof.
- Applying the same hybrid agent to three or more qubits would test whether the normalized QFI ceiling of 1 remains the right target and whether the five-gate solution is a special property of the two-qubit, three-action search space.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes HCQA, a hybrid classical-quantum agent that combines a Deep Q-Network (DQN) with a quantum action-selection circuit, for the automated synthesis of two-qubit quantum sensor circuits (QSCs). The agent selects gates from {Rx, Ry, S} to maximize the Quantum Fisher Information (QFI) while keeping the gate count small. The QFI, computed via Eq. (1) with a threshold T=0.95 and a maximum of 10 actions per episode, is used as the reward. The authors report that the HCQA achieves an average QFI of 1 over 4000 episodes, that episode 3 produces a 5-gate QSC claimed to be an optimal N00N-state design, and that HCQA outperforms the QRA, GAQA, classical DQN, and GPA baselines. The paper also includes hyperparameters, a pseudocode algorithm, and comparisons in Figs. 8 and 9.
Significance. If the claims were fully supported, this would be a modest but useful proof-of-concept showing that a hybrid DQN with a quantum action-selection circuit can synthesize metrologically useful two-qubit sensor circuits. The manuscript provides reproducible implementation details (network architecture, training parameters, pseudocode) and compares against several baselines, which is commendable. However, the central quantitative claim rests on an internally inconsistent QFI normalization, and the action-selection mechanism is described as probabilistic but implemented deterministically. These issues currently undermine the reported 'QFI=1' optimality results and the claimed advantage of quantum action selection. The work is incremental over the authors' prior QRA/GAQA agents, but the hybrid DQN extension is a reasonable direction that can be repaired with corrected metrics and more rigorous evaluation.
major comments (4)
- [Section III-A, Eq. (1)] The QFI normalization is internally inconsistent. Eq. (1) defines QFI = 4(<ψ|Z²|ψ> − <ψ|Z|ψ>²)/n with n=2. For a two-qubit pure state the standard QFI is 4Var(Z); the text states that the raw theoretical maximum is QFI=4, but dividing by n=2 gives a normalized maximum of 2, not 1. To obtain a normalized maximum of 1, the denominator would need to be n²=4. As written, the reward threshold T=0.95 and all reported 'QFI=1' results are calibrated to a metric whose maximum is not 1. This directly affects the central claim that the 5-gate circuit is optimal and that HCQA outperforms other agents. Please correct the normalization and report the raw QFI values for the generated states.
- [Section III-B, Algorithm I and Fig. 3] The action selection is described as producing 'probabilistic action outcomes' that enhance exploration, but the algorithm explicitly selects 'action = argmax(P(a))' after measuring the quantum circuit. This is a deterministic choice. The claimed advantage over classical DQN's deterministic action selection is therefore not realized in the implemented agent; any exploration comes from the epsilon-greedy schedule in the DQN, not from the quantum measurement. Please either implement true sampling from the measured probability distribution or revise the motivation and claims accordingly.
- [Section IV-B, Fig. 9] The comparison with the GPA is not a controlled experiment. The HCQA has already been trained for 4000 episodes (Fig. 8), while the GPA is evaluated for only two episodes on a modified QSC with the S gates removed; the HCQA is also run for only two episodes on this modified task. Two episodes provide no statistical evidence, and the assertion that the maximum QFI is 0.5 in the simplified QSC is given without derivation. Please provide a matched training/evaluation protocol, more episodes, and error bars or statistical significance measures before claiming superiority over the GPA.
- [Section IV-A, Episode 3] The claim that the 5-gate QSC 'represents an optimal design for generating the N00N state' is not supported. The circuit itself is not displayed in the text, and no exhaustive search over gate sequences of length ≤5 or lower bound on gate count for the allowed gate set {Rx, Ry, S} is provided. Without such evidence, 'optimal' is an overstatement. Please show the circuit, verify the generated state overlaps the N00N state, and either provide a brute-force optimality check or soften the claim to 'a compact design found by the agent'.
minor comments (5)
- [Section III-A, Eq. (6)] Equation (6) defines r_i^{DQN} as a target Q-value rather than a reward. This is inconsistent with the pseudocode in Algorithm I, where the reward is the QFI and the target is computed separately. Please clarify the notation to avoid confusion.
- [Section III-B, Figures] The text refers to 'Fig. 2 shows the circuit of the quantum action selection technique' but the action selection circuit appears in Fig. 3. Earlier, Fig. 2 is the QSC structure. Please renumber or correct the cross-references.
- [Section IV-A] The paper says the HCQA 'ran it for three episodes' for the demonstration, but Fig. 8 reports results over 4000 episodes. Please clarify that the three episodes are an illustrative sample and that Fig. 8 is the full training statistics.
- [Section IV-B] In the GPA comparison paragraph, the text says 'The QSC in Fig. 9 is the optimal QSC...' but Fig. 9 is the average QFI plot. It is unclear where this QSC is shown. Please point to the correct figure or include the circuit.
- [Algorithm I] There is a typo: 'Rest the state' should be 'Reset the state'. Additionally, the target update in the pseudocode is written as 'target = Q(s,a) + α(r + γ max Q(s',a'))' while the standard update would be a direct assignment; please check that the intended Q-learning update is stated clearly.
Circularity Check
No significant circularity: QFI is an externally computed metric; the RL loop is standard, and self-citations are baselines, not load-bearing proof.
full rationale
Walking the derivation chain: the HCQA's state representation is the discretized Husimi-Q function (Eq. 3), the DQN outputs Q-values, quantum action selection maps Q-values to rotation angles (Eq. 4), the selected gate is applied to the QSC, and the reward is the QFI computed from the resulting state via Eq. 1. The reported QFI=1 is therefore the training objective evaluated on the actual final state, not a quantity read off from the learned network or defined in terms of the network's own outputs; this is a standard RL training/evaluation loop rather than a circular derivation. The comparisons against QRA, GAQA, and GPA use the authors' own prior agents as baselines; this weakens external benchmarking but is not load-bearing for the central claim, since those baselines are executed and not invoked as proof. The N00N-state optimality is attributed to the external result [26]. There is no uniqueness theorem imported from the authors and no ansatz smuggled in via citation. One substantive concern is the QFI normalization in Section III-A: Eq. 1 with n=2 gives a maximum of 2, not 1, for the stated generator, so the assertion that 'a normalized QFI of 1 corresponds to the theoretical maximum (e.g., QFI = 4 in our two-qubit system)' is internally inconsistent. That is a correctness/calibration issue, not a circularity: the QFI formula is fixed and externally evaluated, and the agent is not fitting a parameter and renaming it a prediction. The Impact Statement also limits the claims to a noise-free two-qubit proof of concept. Overall, no step in the derivation reduces by construction to its own input.
Assumptions & free parameters
free parameters (8)
- DQN learning rate =
0.001
- discount factor gamma =
0.99
- epsilon annealing schedule =
1.0 to 0.01 over 2000 episodes
- replay buffer size =
10000
- mini-batch size =
64
- episode threshold T =
0.95
- max actions per episode =
10
- Husimi-Q discretization grid =
unspecified
assumptions (4)
- domain assumption QFI = 4(Var Z)/n is the correct normalized sensitivity measure with maximum 1 for the N00N state
- domain assumption The N00N state maximizes QFI for this two-qubit QSC with the chosen generator
- domain assumption The gate set {Rx, Ry, S} can express the N00N state within at most 10 gates
- ad hoc to paper The quantum action selection circuit maps Q-values to actions such that argmax probability action maximizes QFI
Cite this review
Pith. "Pith review of HCQA: Hybrid Classical-Quantum Agent for Generating Optimal Quantum Sensor Circuits." pith.science (2026). https://pith.science/paper/F5EQB6YH
@misc{pith2026250821246,
author = {Pith},
title = {Pith review of: HCQA: Hybrid Classical-Quantum Agent for Generating Optimal Quantum Sensor Circuits},
year = {2026},
howpublished = {\url{https://pith.science/paper/F5EQB6YH}},
note = {Machine review of arXiv:2508.21246}
}
read the original abstract
This study proposes an HCQA for designing optimal Quantum Sensor Circuits (QSCs) to address complex quantum physics problems. The HCQA integrates computational intelligence techniques by leveraging a Deep Q-Network (DQN) for learning and policy optimization, enhanced by a quantum-based action selection mechanism based on the Q-values. A quantum circuit encodes the agent current state using Ry gates, and then creates a superposition of possible actions. Measurement of the circuit results in probabilistic action outcomes, allowing the agent to generate optimal QSCs by selecting sequences of gates that maximize the Quantum Fisher Information (QFI) while minimizing the number of gates. This computational intelligence-driven HCQA enables the automated generation of entangled quantum states, specifically the squeezed states, with high QFI sensitivity for quantum state estimation and control. Evaluation of the HCQA on a QSC that consists of two qubits and a sequence of Rx, Ry, and S gates demonstrates its efficiency in generating optimal QSCs with a QFI of 1. This work highlights the synergy between AI-driven learning and quantum computation, illustrating how intelligent agents can autonomously discover optimal quantum circuit designs for enhanced sensing and estimation tasks.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
Quantum Deep Reinforcement Learning for Robot Navigation Tasks
D. Heimann et al., “Quantum Deep Reinforcement Learning for Robot Navigation Tasks”, arXiv:2202.12180, 2022
work page Pith review arXiv 2022
-
[2]
Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning
A. Skolik et al., “Quantum agents in the Gym: a variational quantum algorithm for deep Q-learning”, Phys. Rev, vol 6, doi:10.22331/q-2022- 05-24-720, 2022
doi:10.22331/q-2022- 2022
-
[3]
Robust quantum-inspired reinforcement learning for robot navigation,
D. Dong et al., “Robust quantum-inspired reinforcement learning for robot navigation,” IEEE/ASME Trans. Mechatronics, vol. 17, no. 1, pp. 86–97, 2012, doi: 10.1109/TMECH.2010.2090896
-
[4]
Quantum Reinforcement Learning,
D. Dong et al., "Quantum Reinforcement Learning," IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), vol. 38, no. 5, pp. 1207-1220, 2008, doi: 10.1109/TSMCB.2008.925743
-
[5]
QRA: Quantum Reinforcement Agent for Generating Optimal Quantum Sensor Circuits,
A. Alomari and S. A. P. Kumar, "QRA: Quantum Reinforcement Agent for Generating Optimal Quantum Sensor Circuits," 2024 IEEE International Conference on Quantum Computing and Engineering (QCE), 2024, pp. 1364-1371, doi: 10.1109/QCE60285.2024.00162.C
arXiv 2024
-
[6]
Variational Quantum Circuits for Deep Reinforcement Learning
Samuel et al., “Variational Quantum Circuits for Deep Reinforcement Learning”, Phys. Rev, arXiv:1907.00397v3, 2020
arXiv 1907
-
[7]
ReLAQA: Reinforcement Learning- Based Autonomous Quantum Agent for Quantum Applications,
A. Alomari and S. A. P. Kumar, "ReLAQA: Reinforcement Learning- Based Autonomous Quantum Agent for Quantum Applications," IEEE Transactions on Artificial Intelligence , 2024, doi: 10.1109/TAI.2024.3437335
-
[8]
Quantum multi-agent reinforcement learning via variational quantum circuit design
Yun et al., "Quantum multi-agent reinforcement learning via variational quantum circuit design." In 2022 IEEE 42nd International Conference on Distributed Computing Systems (ICDCS) (pp. 1332-1335), 2022
work page 2022
Show all 32 references
-
[9]
Quantum reinforcement learning in continuous action space,
S. Wu, S. Jin, D. Wen, and X. Wang, "Quantum reinforcement learning in continuous action space," arXiv preprint arXiv:2012.10711, 2020
2012 arXiv
-
[10]
Variational quantum reinforcement learning via evolutionary optimization
Chen, S. Y. C., et al., "Variational quantum reinforcement learning via evolutionary optimization." Machine Learning: Science and Technology, vol. 3, no 2, 2022
2022
-
[11]
Quantum Policy Iteration via Amplitude Estimation and Grover Search Towards Quantum Advantage for Reinforcement Learning,
S. Wiedemann, D. Hein, S. Udluft, and C. B. Mendl, "Quantum Policy Iteration via Amplitude Estimation and Grover Search Towards Quantum Advantage for Reinforcement Learning," arXiv:2206.04741, 2022
2022 arXiv
-
[12]
Measurement-Based Feedback Quantum Control with Deep Reinforcement Learning for a Double-Well Nonlinear Potential,
S. Borah et al., "Measurement-Based Feedback Quantum Control with Deep Reinforcement Learning for a Double-Well Nonlinear Potential," Physical Review Letters, vol. 127, no. 19, 2021
2021
-
[13]
Model-Free Quantum Control with Reinforcement Learning,
V. V. Sivak et al., "Model-Free Quantum Control with Reinforcement Learning," Physical Review X, vol. 12, 2022
2022
-
[14]
Policy gradients using variational quantum circuits,
A. Sequeira, L. P. Santos, and L. S. Barbosa, "Policy gradients using variational quantum circuits," Quantum Machine Intelligence, vol. 5, no. 1, 2023
2023
-
[15]
Squeezed states of light,
D.F. Walls, "Squeezed states of light," Nature, vol. 306, no. 5939, pp. 141-146, 1983
1983
-
[16]
Variational spin-squeezing algorithms on programmable quantum sensors,
R. Kaubruegger et al., "Variational spin-squeezing algorithms on programmable quantum sensors," Physical Review Letters, vol. 123, no. 26, 2019
2019
-
[17]
Optimal metrology with variational quantum circuits on trapped ions,
C. D. Marciniak et al., "Optimal metrology with variational quantum circuits on trapped ions," arXiv preprint arXiv:2107.01860, 2021
2021 arXiv
-
[18]
Quantum variational optimization of Ramsey interferometry and atomic clocks,
R. Kaubruegger, D. V. Vasilyev, M. Schulte, K. Hammerer, and P. Zoller, "Quantum variational optimization of Ramsey interferometry and atomic clocks," Physical Review X, vol. 11, no. 4, 2021
2021
-
[19]
Experiments with separated oscillatory fields and hydrogen masers,
N. F. Ramsey, "Experiments with separated oscillatory fields and hydrogen masers," Rev. Mod. Phys, vol. 62, no. 3, 1990. H. F
1990
-
[20]
Quantum state control of a Bose- Einstein condensate in an optical lattice,
N. Dupont, G. Chatelain, L. Gabardos, M. Arnal, J. Billy, B. Peaudecerf, D. Sugny, and D. Guéry-Odelin, "Quantum state control of a Bose- Einstein condensate in an optical lattice," PRX Quantum, vol. 2, no. 4, 2021
2021
-
[21]
Statistical distance and the geometry of quantum states,
S. L. Braunstein and C. M. Caves, "Statistical distance and the geometry of quantum states," Phys. Rev. Lett., vol. 72, pp. 3439, 1994
1994
-
[22]
Introduction To Quantum Fisher Information,
D. Petz and C. Ghinea, "Introduction To Quantum Fisher Information," World Scientific, pp. 261–281, 2011
2011
-
[23]
Quantum Fisher information from randomized measurements,
A. Rath, C. Branciard, A. Minguzzi, and B. Vermersch, "Quantum Fisher information from randomized measurements," Phys. Rev. Lett., vol. 127, 2021
2021
-
[24]
Quantum Fisher information measurement and verification of the quantum Cramér–Rao bound in a solid-state qubit,
M. Yu, Y. Liu, P. Yang, et al., "Quantum Fisher information measurement and verification of the quantum Cramér–Rao bound in a solid-state qubit," npj Quantum Inf, vol. 8, no. 1, p. 56, 2022. doi: 10.1038/s41534-022-00547-x
2022 doi
-
[25]
Reinforcement-learning-based matter-wave interferometer in a shaken optical lattice,
L. Chih and M. Holland, "Reinforcement-learning-based matter-wave interferometer in a shaken optical lattice," Physical Review Research, vol. 3, no. 3, p. 033279, Sep. 2021. doi: 10.1103/PhysRevResearch.3.033279
2021 doi
-
[26]
A Quantum Rosetta Stone for Interferometry,
H. Lee, P. Kok, and J. P. Dowling, "A Quantum Rosetta Stone for Interferometry," J. Mod. Opt, vol. 49, no. 13, pp. 2325, 2002
2002
-
[27]
Quantum computers: Registers, gates and algorithms,
P. I. Hagouel and I. G. Karafyllidis, "Quantum computers: Registers, gates and algorithms," in 2012 28th International Conference on Microelectronics - Proceedings, MIEL 2012, 2012, pp. 15–21, doi: 10.1109/MIEL.2012.6222789
2012
-
[28]
A Review on quantum computing: From qubits to front-end electronics and cryogenic mosfet physics,
F. Jazaeri, A. Beckers, A. Tajalli, and J. M. Sallese, "A Review on quantum computing: From qubits to front-end electronics and cryogenic mosfet physics," in Proceedings of the 26th International Conference Mixed Design of Integrated Circuits and Systems, MIXDES 2019, 2019, pp...
2019
-
[29]
Poincaré Husimi representation of eigenstates in quantum billiards,
A. Bäcker, S. Fürstberger, and R. Schubert, "Poincaré Husimi representation of eigenstates in quantum billiards," Physical Review E, vol. 70, no. 3, Sep. 2004. doi: 10.1103%2Fphysreve.70.036204
2004
-
[30]
arttner, T. Haas, and O. R. Stockdale,
S. Floerchinger, M. G"arttner, T. Haas, and O. R. Stockdale, "Entropic entanglement criteria in phase space," Phys. Rev. A, vol. 105, no. 1. doi: 10.1103/PhysRevA.105.012409
-
[31]
Quantum convolutional neural networks,
I. Cong, S. Choi, and M. D. Lukin, "Quantum convolutional neural networks," Nature Physics, vol. 15, pp. 1273–1278, 2019, doi: 10.1038/s41567-019-0648-8
2019 doi
-
[32]
GPA: Grover Policy Agent for Generating Optimal Quantum Sensor Circuits,
A. Alomari and S. A. P. Kumar, "GPA: Grover Policy Agent for Generating Optimal Quantum Sensor Circuits," IEEE Transactions on Artificial Intelligence, doi: 10.1109/TAI.2025.3556979
2025
Reviewed August 5, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.