REVIEW 3 major objections 4 minor 21 references
Quantum Architecture Search for Solving Quantum Machine Learning Tasks
T0 review · 3 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read A reinforcement-learning agent can automatically discover compact, high-accuracy parameterized quantum circuit architectures for classification, beating standard hand-designed circuits on Iris and binary MNIST.
desk verdict Competent, incremental RL-QAS for classification, but the reward-uses-test-split setup makes the headline accuracies selected values, not honest generalization estimates. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The engine is a two-loop search-and-evaluate procedure. In the outer loop, a policy-gradient reinforcement learning agent represents a partially built circuit as a three-dimensional binary tensor (qubits by gate-and-control-pair choices by depth layers), picks a gate and target qubits, and updates the tensor. The inner loop trains that circuit's parameters with a classical optimizer on cross-entropy loss and returns its test accuracy. The reward balances a performance term built from test accuracy against a complexity term penalizing remaining depth and gate count, with an extended-horizon boost for later actions; an illegal-action mechanism blocks duplicate gates and depth violations. A has
What would settle it
Hold out a fresh, untouched split of the Iris and binary MNIST data, rerun the search using only training accuracy for reward, and compare the final circuits' accuracy on that split against the standard strongly-entangling-layer baseline; if the RL circuits no longer beat the baseline, the reported advantage came from peeking at the test set.
Extended reading notes
Core claim
The central discovery is that a reinforcement-learning agent exploring a tensor representation of quantum circuits can return architectures that are simultaneously smaller and more accurate than a standard benchmark circuit. For the three-class Iris problem, the best found circuit has four gates (three rotations plus one CNOT) at depth three and reaches 100% test accuracy, while a one-layer strongly-entangling-layer baseline reaches 66% test accuracy. For binary MNIST (digits 0 vs 1, PCA-reduced to 32 features), the best found circuit has six rotation gates and eight CNOTs at depth seven and reaches 91% test accuracy, compared with 77% for the one-layer baseline and 93% for a two-layer basel
Load-bearing premise
The central claim rests on assuming that high test accuracy on two small, balanced, noise-free datasets—Iris and binary MNIST—with the agent selected on those same test splits, is representative of how the strategy would perform across quantum machine learning tasks.
Editorial extensions
If this is right
- Automated architecture search could replace manual ansatz design in quantum machine learning, reducing the human expertise and trial-and-error currently required.
- The discovered circuits' small gate counts and shallow depths make them better suited to noisy near-term quantum hardware than deeper standard architectures.
- Rewarding test accuracy during search can push the agent toward circuits that generalize, not just fit training data.
- Caching circuit evaluations by their structural hash makes the search efficient enough to work despite the exponential size of the architecture space.
- Scaling to harder tasks will require better search stability and evaluation, since the binary-MNIST run did not converge.
Reading between the lines
- Because the same 30% test split is used both to compute the reward and to select the final circuit, the reported test accuracies are likely optimistic; a fair evaluation would hold out a third split for reporting.
- The recurring 'CNOT early, rotations later' pattern could be extracted as a prior for human ansatz design or to seed other search methods, though the paper does not claim it as a general design rule.
- A direct next test would apply the same search to a larger or noisy classification task and compare against random search with an equal evaluation budget; the viability claim would be much stronger if that comparison holds.
- Replacing the full training of every candidate with a learned performance predictor could address the scaling bottleneck the paper identifies for more complex tasks.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes RL-QAS, a two-loop reinforcement-learning framework for quantum architecture search (QAS) applied to classification. An outer-loop PPO agent incrementally constructs parameterized quantum circuit architectures (PQCAs), while an inner loop trains and evaluates each candidate on a quantum machine learning task. The reward is a weighted combination of test accuracy and circuit complexity (Eq. 3). The framework is evaluated on Iris (binary and 3-class) and binary MNIST (0/1), and compared against random and strongly-entangling-layer baselines. The authors report that RL-QAS finds low-complexity circuits with 100% test accuracy on Iris and 91% on MNIST 2, and conclude that RL is a viable approach for QAS in QML. The paper includes code, training metadata, and architectural analyses.
Significance. If the reported results were unbiased, the paper would make a modest but useful contribution: it is among the first to apply RL-based QAS to classification tasks, and it provides a reusable framework, caching mechanism, and detailed architectural pattern analysis. The decoupled outer/inner loop and the complexity-aware reward are sensible design choices. However, the central empirical claim is weakened by a selection-bias problem (test accuracy used both as reward and as final report) and by the very limited benchmark scope. The paper is therefore best viewed as a promising feasibility study whose main quantitative claims need re-validation before the viability conclusion can be accepted.
major comments (3)
- [Experimental Setup / Eq. (3), Table 3] The reward in Eq. (3) is computed from test accuracy on the fixed 30% test split, and Table 3 reports TeA on that same split. With over 9,000 unique PQCAs evaluated for Iris (§Macro-Analysis), the reported 100% TeA is the maximum of a large set of evaluations on the test set, not an unbiased estimate of generalization. There is no separate validation set. This selection bias inflates the headline accuracies and directly undermines the central claim of 'high test accuracy'. Please introduce a held-out test set used only for final reporting, use a validation set for reward/selection, and report mean/standard error over seeds/architectures (or at least the full distribution).
- [Table 2 / Discussion] The claim that RL is a viable approach for QAS in QML is supported almost entirely by two small datasets. The MNIST 2 agent did not converge (Table 2), and the best MNIST 2 TeA in Table 3 comes from a non-converged run. Please either add more diverse benchmarks with statistical significance tests, or substantially weaken the conclusion to a feasibility study. At minimum, report metrics over the 3 seeds rather than only the best circuit.
- [Results (Performance of the RL-QAS Agent)] The text repeatedly states that RL-QAS outperforms a random baseline, but no quantitative comparison is provided in Table 3 or the main text. Since the random baseline is one of the two main baselines used to support the viability claim, please include its accuracy/complexity numbers or clearly reference the supplementary figure where these data appear.
minor comments (4)
- [Eq. (4)] The second line of Eq. (4) uses 'r=' but should define P_delta; this is likely a typographical copy-paste error.
- [Table 2] The table uses inconsistent decimal separators (e.g., 11,87 vs 11.84) and contains a stray '1' at the bottom; please format uniformly.
- [Related Work] 'This thesis addresses this gap' should be 'This paper addresses this gap'.
- [Experimental Setup] The statement that computing the reward from test accuracy 'promotes generalization' is misleading; using test data in the reward is a form of selection on the test set. Please rephrase and address the issue directly (see major comment 1).
Circularity Check
Reported test accuracies are the optimized reward itself: the RL agent maximizes test accuracy on the fixed 30% split and the same split is then reported as TeA.
-
fitted input called prediction
[Experimental Setup (Measurement and Post-processing); Eq. (3); Results Table 3]
"To promote generalization, the reward is calculated based solely on test accuracy. This encourages the RL-QAS agent to construct PQCAs that generalize well rather than overfitting the training data."
Eq. (3) computes the reward from P_delta, where P_delta is P_current, the accuracy performance of the PQCA. The Experimental Setup states this accuracy is the test accuracy on the fixed 30% split (70/30 train/test). The agent maximizes this reward over 9,000+ unique PQCAs (Macro-Analysis), and Table 3 reports the best TeA (e.g., 1.0 Iris, 0.91 MNIST 2) on the same split. Therefore the reported TeA is the selection objective, not an unbiased out-of-sample estimate: the 'high test accuracy' claim is by construction the value being optimized, so the result reduces to a fitted input called a prediction. No separate validation set protects the reported TeA from selection bias.
full rationale
The central empirical evidence for RL viability is Table 3's TeA values. Those values are computed on the same fixed 30% test split that serves as the reward signal in Eq. (3). Because the RL agent searches over thousands of architectures and is selected by that reward, the reported TeA is the maximum of many test-set evaluations rather than an independent generalization estimate. This is a partial circularity: the 'high test accuracy' result is built into the optimization target. The paper itself flags that 'performance evaluation mechanisms' need refinement (Abstract/Conclusion), consistent with this concern. There is no equation-level self-identity (Eq. X = Eq. Y) and no load-bearing self-citation: citations to prior work by the same authors (Kölle et al. 2024, Altmann et al. 2024) are for implementation details, not for the core claim. The search/learning component retains independent content — the agent does learn to propose circuits and to reduce complexity — but the headline accuracy numbers are compromised by selection on the test split, so the score is 6 rather than 0-2.
Assumptions & free parameters
free parameters (6)
- PPO hyperparameters (learning rate 0.003, entropy coefficient 0.03, n_steps, batch size) =
learning rate 0.003, ent coeff 0.03, n_steps 128/512/1024, batch size 128
- Inner-loop optimizer learning rate and parameter initialization range =
Adam lr 0.01, init range [-1.0, 1.0]
- Reward shaping coefficients (0.1, complexity weights, E_H = Depth_max * 10, penalty -0.01, bonus +100) =
0.1, 1/2, 1/2, Depth_max*10, -0.01, +100
- Maximum circuit depth per task =
4-7 depending on task
- PCA dimensionality reduction to 32 features for MNIST =
32 features (97.6% variance retained)
- Gate set and encoding scheme (amplitude encoding, {Rx, Ry, Rz, CNOT}) =
fixed gate set and amplitude encoding
assumptions (4)
- domain assumption Noise-free simulation is an adequate proxy for quantum circuit performance on NISQ devices.
- domain assumption Amplitude encoding with L2 normalization preserves the classification information of Iris and MNIST features.
- domain assumption Test accuracy on a fixed 30% split is a valid reward signal for generalization.
- domain assumption PPO (Stable Baselines3/JAX implementation) behaves as a standard, sufficiently converged RL optimizer for this task.
Cite this review
Pith. "Pith review of Quantum Architecture Search for Solving Quantum Machine Learning Tasks." pith.science (2026). https://pith.science/paper/GDMHT3B7
@misc{pith2026250911198,
author = {Pith},
title = {Pith review of: Quantum Architecture Search for Solving Quantum Machine Learning Tasks},
year = {2026},
howpublished = {\url{https://pith.science/paper/GDMHT3B7}},
note = {Machine review of arXiv:2509.11198}
}
read the original abstract
Quantum computing leverages quantum mechanics to address computational problems in ways that differ fundamentally from classical approaches. While current quantum hardware remains error-prone and limited in scale, Variational Quantum Circuits offer a noise-resilient framework suitable for today's devices. The performance of these circuits strongly depends on the underlying architecture of their parameterized quantum components. Identifying efficient, hardware-compatible quantum circuit architectures -- known as Quantum Architecture Search (QAS) -- is therefore essential. Manual QAS is complex and error-prone, motivating efforts to automate it. Among various automated strategies, Reinforcement Learning (RL) remains underexplored, particularly in Quantum Machine Learning contexts. This work introduces RL-QAS, a framework that applies RL to discover effective circuit architectures for classification tasks. We evaluate RL-QAS using the Iris and binary MNIST datasets. The agent autonomously discovers low-complexity circuit designs that achieve high test accuracy. Our results show that RL is a viable approach for automated architecture search in quantum machine learning. However, applying RL-QAS to more complex tasks will require further refinement of the search strategy and performance evaluation mechanisms.
Figures
Figures from the paper (15 more)
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
o lle, M.; B \
Altmann, P.; Stein, J.; K \"o lle, M.; B \"a rligea, A.; Zorn, M.; Gabor, T.; Phan, T.; Feld, S.; and Linnhoff-Popien, C. 2024. Challenges for Reinforcement Learning in Quantum Circuit Design. In 2024 IEEE International Conference on Quantum Computing and Engineering (QCE), volume 1, 1600--1610. IEEE
2024
-
[4]
Chivilikhin, D.; Samarin, A.; Ulyantsev, V.; Iorsh, I.; Oganov, A. R.; and Kyriienko, O. 2020. MoG-VQE: Multiobjective genetic variational quantum eigensolver. arXiv preprint arXiv:2007.04424
arXiv 2020
-
[5]
He, Z.; Deng, M.; Zheng, S.; Li, L.; and Situ, H. 2023. Gsqas: graph self-supervised quantum architecture search. Physica A: Statistical Mechanics and its Applications, 630: 129286
2023
-
[6]
Huang, Y.; Li, Q.; Hou, X.; Wu, R.; Yung, M.-H.; Bayat, A.; and Wang, X. 2022. Robust resource-efficient quantum variational ansatz through an evolutionary algorithm. Physical Review A, 105(5): 052414
2022
-
[7]
K \"o lle, M.; Schubert, T.; Altmann, P.; Zorn, M.; Stein, J.; and Linnhoff-Popien, C. 2024. A reinforcement learning environment for directed quantum circuit synthesis. arXiv preprint arXiv:2401.07054
arXiv 2024
-
[8]
Kundu, A. 2024. Reinforcement learning-assisted quantum architecture search for variational quantum algorithms. arXiv preprint arXiv:2402.13754
arXiv 2024
Show all 21 references
-
[9]
J.; Dunjko, V.; and Miszczak, J
Kundu, A.; Bede ek, P.; Ostaszewski, M.; Danaci, O.; Patel, Y. J.; Dunjko, V.; and Miszczak, J. A. 2024. Enhancing variational quantum state diagonalization using reinforcement learning techniques. New Journal of Physics, 26(1): 013034
2024
-
[10]
Lu, Z.; Shen, P.-X.; and Deng, D.-L. 2021. Markovian quantum neuroevolution for machine learning. Physical Review Applied, 16(4): 044039
2021
-
[11]
A.; Davis, E.; Alam, M
McKiernan, K. A.; Davis, E.; Alam, M. S.; and Rigetti, C. 2019. Automated quantum programming via reinforcement learning for combinatorial optimization. arXiv preprint arXiv:1908.08054
2019 arXiv
-
[12]
Meng, F.-X.; Li, Z.-T.; Yu, X.-T.; and Zhang, Z.-C. 2021. Quantum circuit architecture optimization for variational quantum eigensolver via monto carlo tree search. IEEE Transactions on Quantum Engineering, 2: 1--10
2021
-
[13]
M.; Masarczyk, W.; Scerri, E.; and Dunjko, V
Ostaszewski, M.; Trenkwalder, L. M.; Masarczyk, W.; Scerri, E.; and Dunjko, V. 2021. Reinforcement learning for optimization of variational quantum circuit architectures. Advances in neural information processing systems, 34: 18182--18194
2021
-
[14]
J.; Kundu, A.; Ostaszewski, M.; Bonet-Monroig, X.; Dunjko, V.; and Danaci, O
Patel, Y. J.; Kundu, A.; Ostaszewski, M.; Bonet-Monroig, X.; Dunjko, V.; and Danaci, O. 2024. Curriculum reinforcement learning for quantum architecture search under hardware errors. arXiv preprint arXiv:2402.03500
2024 arXiv
-
[15]
G.; Hu, S.; Pistoia, M.; Chen, R.; and Wood, S
Rattew, A. G.; Hu, S.; Pistoia, M.; Chen, R.; and Wood, S. 2019. A domain-agnostic, noise-resistant, hardware-efficient evolutionary variational quantum eigensolver. arXiv preprint arXiv:1910.09694
2019 arXiv
-
[16]
Z.; Chong, F
Wang, H.; Ding, Y.; Gu, J.; Lin, Y.; Pan, D. Z.; Chong, F. T.; and Han, S. 2022. Quantumnas: Noise-adaptive search for robust quantum circuits. In 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA), 692--708. IEEE
2022
-
[17]
C.; and Myers, C
Wang, P.; Usman, M.; Parampalli, U.; Hollenberg, L. C.; and Myers, C. R. 2023. Automated quantum circuit design with nested monte carlo tree search. IEEE Transactions on Quantum Engineering, 4: 1--20
2023
-
[18]
Wu, W.; Yan, G.; Lu, X.; Pan, K.; and Yan, J. 2023. Quantumdarts: differentiable quantum architecture search for variational quantum algorithms. In International conference on machine learning, 37745--37764. PMLR
2023
-
[19]
Yao, J.; Li, H.; Bukov, M.; Lin, L.; and Ying, L. 2022. Monte carlo tree search based hybrid optimization of variational quantum circuits. In Mathematical and Scientific Machine Learning, 49--64. PMLR
2022
-
[20]
Zhang, S.-X.; Hsieh, C.-Y.; Zhang, S.; and Yao, H. 2021. Neural predictor based quantum architecture search. Machine Learning: Science and Technology, 2(4): 045027
2021
-
[21]
Zhang, S.-X.; Hsieh, C.-Y.; Zhang, S.; and Yao, H. 2022. Differentiable quantum architecture search. Quantum Science and Technology, 7(4): 045023
2022
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.