Pith. sign in

REVIEW 3 major objections 4 minor 21 references

Quantum Architecture Search for Solving Quantum Machine Learning Tasks

T0 review · 3 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Pith's one-line read A reinforcement-learning agent can automatically discover compact, high-accuracy parameterized quantum circuit architectures for classification, beating standard hand-designed circuits on Iris and binary MNIST.

desk verdict Competent, incremental RL-QAS for classification, but the reward-uses-test-split setup makes the headline accuracies selected values, not honest generalization estimates. read the letter →

arxiv 2509.11198 v1 pith:GDMHT3B7 submitted 2025-09-14 quant-ph cs.AIcs.LG

classification quant-phcs.AIcs.LG
keywords quantumarchitecturesearchreinforcementlearningvariationalcircuitsmachinecircuitdesignclassificationIrisdatasetMNIST
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to show that reinforcement learning can automate the design of variational quantum circuits for machine-learning tasks, a job normally done by hand. The agent builds circuits gate by gate, and each candidate is trained and scored on a classification task. On the full Iris dataset, the best discovered circuit—four gates including one CNOT, depth three—reaches 100% test accuracy; on binary MNIST, a circuit with six rotations and eight CNOTs reaches 91% test accuracy. The authors argue this makes RL a viable strategy for quantum architecture search, and that the resulting low-depth circuits are well suited to near-term quantum hardware. If correct, automated search could lower the expertise barrier for designing quantum machine-learning models.

What carries the argument

The engine is a two-loop search-and-evaluate procedure. In the outer loop, a policy-gradient reinforcement learning agent represents a partially built circuit as a three-dimensional binary tensor (qubits by gate-and-control-pair choices by depth layers), picks a gate and target qubits, and updates the tensor. The inner loop trains that circuit's parameters with a classical optimizer on cross-entropy loss and returns its test accuracy. The reward balances a performance term built from test accuracy against a complexity term penalizing remaining depth and gate count, with an extended-horizon boost for later actions; an illegal-action mechanism blocks duplicate gates and depth violations. A has

What would settle it

Hold out a fresh, untouched split of the Iris and binary MNIST data, rerun the search using only training accuracy for reward, and compare the final circuits' accuracy on that split against the standard strongly-entangling-layer baseline; if the RL circuits no longer beat the baseline, the reported advantage came from peeking at the test set.

Watch

Extended reading notes

Core claim

The central discovery is that a reinforcement-learning agent exploring a tensor representation of quantum circuits can return architectures that are simultaneously smaller and more accurate than a standard benchmark circuit. For the three-class Iris problem, the best found circuit has four gates (three rotations plus one CNOT) at depth three and reaches 100% test accuracy, while a one-layer strongly-entangling-layer baseline reaches 66% test accuracy. For binary MNIST (digits 0 vs 1, PCA-reduced to 32 features), the best found circuit has six rotation gates and eight CNOTs at depth seven and reaches 91% test accuracy, compared with 77% for the one-layer baseline and 93% for a two-layer basel

Load-bearing premise

The central claim rests on assuming that high test accuracy on two small, balanced, noise-free datasets—Iris and binary MNIST—with the agent selected on those same test splits, is representative of how the strategy would perform across quantum machine learning tasks.

Editorial extensions

If this is right

  • Automated architecture search could replace manual ansatz design in quantum machine learning, reducing the human expertise and trial-and-error currently required.
  • The discovered circuits' small gate counts and shallow depths make them better suited to noisy near-term quantum hardware than deeper standard architectures.
  • Rewarding test accuracy during search can push the agent toward circuits that generalize, not just fit training data.
  • Caching circuit evaluations by their structural hash makes the search efficient enough to work despite the exponential size of the architecture space.
  • Scaling to harder tasks will require better search stability and evaluation, since the binary-MNIST run did not converge.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the same 30% test split is used both to compute the reward and to select the final circuit, the reported test accuracies are likely optimistic; a fair evaluation would hold out a third split for reporting.
  • The recurring 'CNOT early, rotations later' pattern could be extracted as a prior for human ansatz design or to seed other search methods, though the paper does not claim it as a general design rule.
  • A direct next test would apply the same search to a larger or noisy classification task and compare against random search with an equal evaluation budget; the viability claim would be much stronger if that comparison holds.
  • Replacing the full training of every candidate with a learned performance predictor could address the scaling bottleneck the paper identifies for more complex tasks.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The manuscript proposes RL-QAS, a two-loop reinforcement-learning framework for quantum architecture search (QAS) applied to classification. An outer-loop PPO agent incrementally constructs parameterized quantum circuit architectures (PQCAs), while an inner loop trains and evaluates each candidate on a quantum machine learning task. The reward is a weighted combination of test accuracy and circuit complexity (Eq. 3). The framework is evaluated on Iris (binary and 3-class) and binary MNIST (0/1), and compared against random and strongly-entangling-layer baselines. The authors report that RL-QAS finds low-complexity circuits with 100% test accuracy on Iris and 91% on MNIST 2, and conclude that RL is a viable approach for QAS in QML. The paper includes code, training metadata, and architectural analyses.

Significance. If the reported results were unbiased, the paper would make a modest but useful contribution: it is among the first to apply RL-based QAS to classification tasks, and it provides a reusable framework, caching mechanism, and detailed architectural pattern analysis. The decoupled outer/inner loop and the complexity-aware reward are sensible design choices. However, the central empirical claim is weakened by a selection-bias problem (test accuracy used both as reward and as final report) and by the very limited benchmark scope. The paper is therefore best viewed as a promising feasibility study whose main quantitative claims need re-validation before the viability conclusion can be accepted.

major comments (3)
  1. [Experimental Setup / Eq. (3), Table 3] The reward in Eq. (3) is computed from test accuracy on the fixed 30% test split, and Table 3 reports TeA on that same split. With over 9,000 unique PQCAs evaluated for Iris (§Macro-Analysis), the reported 100% TeA is the maximum of a large set of evaluations on the test set, not an unbiased estimate of generalization. There is no separate validation set. This selection bias inflates the headline accuracies and directly undermines the central claim of 'high test accuracy'. Please introduce a held-out test set used only for final reporting, use a validation set for reward/selection, and report mean/standard error over seeds/architectures (or at least the full distribution).
  2. [Table 2 / Discussion] The claim that RL is a viable approach for QAS in QML is supported almost entirely by two small datasets. The MNIST 2 agent did not converge (Table 2), and the best MNIST 2 TeA in Table 3 comes from a non-converged run. Please either add more diverse benchmarks with statistical significance tests, or substantially weaken the conclusion to a feasibility study. At minimum, report metrics over the 3 seeds rather than only the best circuit.
  3. [Results (Performance of the RL-QAS Agent)] The text repeatedly states that RL-QAS outperforms a random baseline, but no quantitative comparison is provided in Table 3 or the main text. Since the random baseline is one of the two main baselines used to support the viability claim, please include its accuracy/complexity numbers or clearly reference the supplementary figure where these data appear.
minor comments (4)
  1. [Eq. (4)] The second line of Eq. (4) uses 'r=' but should define P_delta; this is likely a typographical copy-paste error.
  2. [Table 2] The table uses inconsistent decimal separators (e.g., 11,87 vs 11.84) and contains a stray '1' at the bottom; please format uniformly.
  3. [Related Work] 'This thesis addresses this gap' should be 'This paper addresses this gap'.
  4. [Experimental Setup] The statement that computing the reward from test accuracy 'promotes generalization' is misleading; using test data in the reward is a form of selection on the test set. Please rephrase and address the issue directly (see major comment 1).

Circularity Check

1 steps flagged · score 6.0 of 10

Reported test accuracies are the optimized reward itself: the RL agent maximizes test accuracy on the fixed 30% split and the same split is then reported as TeA.

  1. fitted input called prediction [Experimental Setup (Measurement and Post-processing); Eq. (3); Results Table 3]
    "To promote generalization, the reward is calculated based solely on test accuracy. This encourages the RL-QAS agent to construct PQCAs that generalize well rather than overfitting the training data."

    Eq. (3) computes the reward from P_delta, where P_delta is P_current, the accuracy performance of the PQCA. The Experimental Setup states this accuracy is the test accuracy on the fixed 30% split (70/30 train/test). The agent maximizes this reward over 9,000+ unique PQCAs (Macro-Analysis), and Table 3 reports the best TeA (e.g., 1.0 Iris, 0.91 MNIST 2) on the same split. Therefore the reported TeA is the selection objective, not an unbiased out-of-sample estimate: the 'high test accuracy' claim is by construction the value being optimized, so the result reduces to a fitted input called a prediction. No separate validation set protects the reported TeA from selection bias.

full rationale

The central empirical evidence for RL viability is Table 3's TeA values. Those values are computed on the same fixed 30% test split that serves as the reward signal in Eq. (3). Because the RL agent searches over thousands of architectures and is selected by that reward, the reported TeA is the maximum of many test-set evaluations rather than an independent generalization estimate. This is a partial circularity: the 'high test accuracy' result is built into the optimization target. The paper itself flags that 'performance evaluation mechanisms' need refinement (Abstract/Conclusion), consistent with this concern. There is no equation-level self-identity (Eq. X = Eq. Y) and no load-bearing self-citation: citations to prior work by the same authors (Kölle et al. 2024, Altmann et al. 2024) are for implementation details, not for the core claim. The search/learning component retains independent content — the agent does learn to propose circuits and to reduce complexity — but the headline accuracy numbers are compromised by selection on the test split, so the score is 6 rather than 0-2.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

No new physical entities, forces, or conserved quantities are introduced. The free-parameter count is dominated by hand-chosen reward coefficients and hyperparameters, plus the fixed encoding/gate-set choices. The most consequential modeling choice is the use of test accuracy as the reward, which directly shapes the main empirical claim. The axioms are standard domain assumptions for this type of variational-circuit simulation study.

free parameters (6)
  • PPO hyperparameters (learning rate 0.003, entropy coefficient 0.03, n_steps, batch size) = learning rate 0.003, ent coeff 0.03, n_steps 128/512/1024, batch size 128
    Selected by manual grid search over learning rate, entropy coefficient, n_steps, and batch size. Reported performance depends on these choices, and the paper explicitly says automated tuning was not feasible.
  • Inner-loop optimizer learning rate and parameter initialization range = Adam lr 0.01, init range [-1.0, 1.0]
    Chosen from explored ranges [-0.5,0.5], [-1,1], [-2,2], [-pi,pi]; the final choice affects VQC training and therefore the reward signal.
  • Reward shaping coefficients (0.1, complexity weights, E_H = Depth_max * 10, penalty -0.01, bonus +100) = 0.1, 1/2, 1/2, Depth_max*10, -0.01, +100
    Hand-chosen weights in the reward function (Eq. 3); no sensitivity analysis is provided, and the agent's discovered circuits depend on these choices.
  • Maximum circuit depth per task = 4-7 depending on task
    Treated as a tunable hyperparameter limiting the search space; different depths produce different best circuits.
  • PCA dimensionality reduction to 32 features for MNIST = 32 features (97.6% variance retained)
    A preprocessing choice that reduces the qubit count and action space; the reported MNIST accuracy depends on this projection.
  • Gate set and encoding scheme (amplitude encoding, {Rx, Ry, Rz, CNOT}) = fixed gate set and amplitude encoding
    The search is restricted to this gate set and encoding; the paper states encoding is fixed and not controlled by the agent. This is a modeling choice that constrains the claim.
assumptions (4)
  • domain assumption Noise-free simulation is an adequate proxy for quantum circuit performance on NISQ devices.
    The paper evaluates on noise-free PennyLane simulators and states that hardware evaluation is future work. The conclusion that RL-QAS finds hardware-compatible compact circuits assumes simulation accuracy transfers to real devices.
  • domain assumption Amplitude encoding with L2 normalization preserves the classification information of Iris and MNIST features.
    Introduced in the Dataset and Encoding section. The encoding is fixed and not searched over; the validity of the results depends on this encoding retaining enough information, which is plausible but not established.
  • domain assumption Test accuracy on a fixed 30% split is a valid reward signal for generalization.
    The reward is computed solely from test accuracy, so the RL agent directly optimizes that quantity. This assumes the fixed split is stable and that optimizing it does not overfit the test set.
  • domain assumption PPO (Stable Baselines3/JAX implementation) behaves as a standard, sufficiently converged RL optimizer for this task.
    The framework relies on PPO for the outer loop; no proof of convergence or coverage of the architecture space is given, and for MNIST the agent 'did not converge' by the authors' own report.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Quantum Architecture Search for Solving Quantum Machine Learning Tasks." pith.science (2026). https://pith.science/paper/GDMHT3B7

@misc{pith2026250911198,
  author       = {Pith},
  title        = {Pith review of: Quantum Architecture Search for Solving Quantum Machine Learning Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GDMHT3B7}},
  note         = {Machine review of arXiv:2509.11198}
}
read the original abstract

Quantum computing leverages quantum mechanics to address computational problems in ways that differ fundamentally from classical approaches. While current quantum hardware remains error-prone and limited in scale, Variational Quantum Circuits offer a noise-resilient framework suitable for today's devices. The performance of these circuits strongly depends on the underlying architecture of their parameterized quantum components. Identifying efficient, hardware-compatible quantum circuit architectures -- known as Quantum Architecture Search (QAS) -- is therefore essential. Manual QAS is complex and error-prone, motivating efforts to automate it. Among various automated strategies, Reinforcement Learning (RL) remains underexplored, particularly in Quantum Machine Learning contexts. This work introduces RL-QAS, a framework that applies RL to discover effective circuit architectures for classification tasks. We evaluate RL-QAS using the Iris and binary MNIST datasets. The agent autonomously discovers low-complexity circuit designs that achieve high test accuracy. Our results show that RL is a viable approach for automated architecture search in quantum machine learning. However, applying RL-QAS to more complex tasks will require further refinement of the search strategy and performance evaluation mechanisms.

Figures

Figures reproduced from arXiv: 2509.11198 by the authors.

Figure 1
Figure 1. Overall architecture of the RL-QAS framework. The outer loop constructs PQCAs; the inner loop evaluates their [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Tensor-based PQCA encoding with a binary 3D [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Training performance of the RL-QAS agent for Iris using test accuracy, episode reward, number of gates and circuit [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (15 more)
Figure 4
Figure 4. Figure 4: Architecture of the SEL PQCA used for bench [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 6
Figure 6. Figure 6: Analysis of gate pair transitions and gate usage by [PITH_FULL_IMAGE:figures/full_fig_p006_6.png]
Figure 5
Figure 5. Figure 5: Comparison of PQCAs in terms of test accuracy, [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 7
Figure 7. Figure 7: Best PQCAs for binary Iris, Iris, and MNIST 2 classification. [PITH_FULL_IMAGE:figures/full_fig_p007_7.png]
Figure 8
Figure 8. Figure 8: Cost landscape of the best PQCA for Iris across [PITH_FULL_IMAGE:figures/full_fig_p007_8.png]
Figure 9
Figure 9. Figure 9: Achieved (a) accuracy and (b) reward as a function of the completed training steps for the Iris 2 (0, 1) classification [PITH_FULL_IMAGE:figures/full_fig_p009_9.png]
Figure 10
Figure 10. Figure 10: Number of (a) quantum gates used and (b) circuit depth utilized as a function of the completed training steps for the [PITH_FULL_IMAGE:figures/full_fig_p009_10.png]
Figure 11
Figure 11. Figure 11: Achieved (a) accuracy and (b) reward as a function of the completed training steps for the Iris 2 (0, 2) classification [PITH_FULL_IMAGE:figures/full_fig_p009_11.png]
Figure 12
Figure 12. Figure 12: Number of (a) quantum gates used and (b) circuit depth utilized as a function of the completed training steps for the [PITH_FULL_IMAGE:figures/full_fig_p010_12.png]
Figure 13
Figure 13. Figure 13: Achieved (a) accuracy and (b) reward as a function of the completed training steps for the Iris 2 (1, 2) classification [PITH_FULL_IMAGE:figures/full_fig_p010_13.png]
Figure 14
Figure 14. Figure 14: Number of (a) quantum gates used and (b) circuit depth utilized as a function of the completed training steps for the [PITH_FULL_IMAGE:figures/full_fig_p010_14.png]
Figure 15
Figure 15. Figure 15: Achieved (a) accuracy and (b) reward as a function of the completed training steps for the MNIST 2 classification [PITH_FULL_IMAGE:figures/full_fig_p011_15.png]
Figure 16
Figure 16. Figure 16: Number of (a) quantum gates used and (b) circuit depth utilized as a function of the completed training steps for the [PITH_FULL_IMAGE:figures/full_fig_p011_16.png]
Figure 17
Figure 17. Figure 17: Optimization behavior of the best PQCA found within the RL-QAS for the Iris 2 classification problems [PITH_FULL_IMAGE:figures/full_fig_p011_17.png]
Figure 18
Figure 18. Figure 18: Optimization behavior of the best PQCA found within the RL-QAS compared to a SEL VQC with one layer for the [PITH_FULL_IMAGE:figures/full_fig_p012_18.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

21 extracted references · 6 linked inside Pith

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    o lle, M.; B \

    Altmann, P.; Stein, J.; K \"o lle, M.; B \"a rligea, A.; Zorn, M.; Gabor, T.; Phan, T.; Feld, S.; and Linnhoff-Popien, C. 2024. Challenges for Reinforcement Learning in Quantum Circuit Design. In 2024 IEEE International Conference on Quantum Computing and Engineering (QCE), volume 1, 1600--1610. IEEE

  4. [4]

    R.; and Kyriienko, O

    Chivilikhin, D.; Samarin, A.; Ulyantsev, V.; Iorsh, I.; Oganov, A. R.; and Kyriienko, O. 2020. MoG-VQE: Multiobjective genetic variational quantum eigensolver. arXiv preprint arXiv:2007.04424

  5. [5]

    He, Z.; Deng, M.; Zheng, S.; Li, L.; and Situ, H. 2023. Gsqas: graph self-supervised quantum architecture search. Physica A: Statistical Mechanics and its Applications, 630: 129286

  6. [6]

    Huang, Y.; Li, Q.; Hou, X.; Wu, R.; Yung, M.-H.; Bayat, A.; and Wang, X. 2022. Robust resource-efficient quantum variational ansatz through an evolutionary algorithm. Physical Review A, 105(5): 052414

  7. [7]

    K \"o lle, M.; Schubert, T.; Altmann, P.; Zorn, M.; Stein, J.; and Linnhoff-Popien, C. 2024. A reinforcement learning environment for directed quantum circuit synthesis. arXiv preprint arXiv:2401.07054

  8. [8]

    Kundu, A. 2024. Reinforcement learning-assisted quantum architecture search for variational quantum algorithms. arXiv preprint arXiv:2402.13754

Show all 21 references
  1. [9]

    J.; Dunjko, V.; and Miszczak, J

    Kundu, A.; Bede ek, P.; Ostaszewski, M.; Danaci, O.; Patel, Y. J.; Dunjko, V.; and Miszczak, J. A. 2024. Enhancing variational quantum state diagonalization using reinforcement learning techniques. New Journal of Physics, 26(1): 013034

  2. [10]

    Lu, Z.; Shen, P.-X.; and Deng, D.-L. 2021. Markovian quantum neuroevolution for machine learning. Physical Review Applied, 16(4): 044039

  3. [11]

    A.; Davis, E.; Alam, M

    McKiernan, K. A.; Davis, E.; Alam, M. S.; and Rigetti, C. 2019. Automated quantum programming via reinforcement learning for combinatorial optimization. arXiv preprint arXiv:1908.08054

  4. [12]

    Meng, F.-X.; Li, Z.-T.; Yu, X.-T.; and Zhang, Z.-C. 2021. Quantum circuit architecture optimization for variational quantum eigensolver via monto carlo tree search. IEEE Transactions on Quantum Engineering, 2: 1--10

  5. [13]

    M.; Masarczyk, W.; Scerri, E.; and Dunjko, V

    Ostaszewski, M.; Trenkwalder, L. M.; Masarczyk, W.; Scerri, E.; and Dunjko, V. 2021. Reinforcement learning for optimization of variational quantum circuit architectures. Advances in neural information processing systems, 34: 18182--18194

  6. [14]

    J.; Kundu, A.; Ostaszewski, M.; Bonet-Monroig, X.; Dunjko, V.; and Danaci, O

    Patel, Y. J.; Kundu, A.; Ostaszewski, M.; Bonet-Monroig, X.; Dunjko, V.; and Danaci, O. 2024. Curriculum reinforcement learning for quantum architecture search under hardware errors. arXiv preprint arXiv:2402.03500

  7. [15]

    G.; Hu, S.; Pistoia, M.; Chen, R.; and Wood, S

    Rattew, A. G.; Hu, S.; Pistoia, M.; Chen, R.; and Wood, S. 2019. A domain-agnostic, noise-resistant, hardware-efficient evolutionary variational quantum eigensolver. arXiv preprint arXiv:1910.09694

  8. [16]

    Z.; Chong, F

    Wang, H.; Ding, Y.; Gu, J.; Lin, Y.; Pan, D. Z.; Chong, F. T.; and Han, S. 2022. Quantumnas: Noise-adaptive search for robust quantum circuits. In 2022 IEEE International Symposium on High-Performance Computer Architecture (HPCA), 692--708. IEEE

  9. [17]

    C.; and Myers, C

    Wang, P.; Usman, M.; Parampalli, U.; Hollenberg, L. C.; and Myers, C. R. 2023. Automated quantum circuit design with nested monte carlo tree search. IEEE Transactions on Quantum Engineering, 4: 1--20

  10. [18]

    Wu, W.; Yan, G.; Lu, X.; Pan, K.; and Yan, J. 2023. Quantumdarts: differentiable quantum architecture search for variational quantum algorithms. In International conference on machine learning, 37745--37764. PMLR

  11. [19]

    Yao, J.; Li, H.; Bukov, M.; Lin, L.; and Ying, L. 2022. Monte carlo tree search based hybrid optimization of variational quantum circuits. In Mathematical and Scientific Machine Learning, 49--64. PMLR

  12. [20]

    Zhang, S.-X.; Hsieh, C.-Y.; Zhang, S.; and Yao, H. 2021. Neural predictor based quantum architecture search. Machine Learning: Science and Technology, 2(4): 045027

  13. [21]

    Zhang, S.-X.; Hsieh, C.-Y.; Zhang, S.; and Yao, H. 2022. Differentiable quantum architecture search. Quantum Science and Technology, 7(4): 045023

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.