Pith. sign in

REVIEW 3 major objections 3 minor 48 references

Unitary Synthesis with AlphaZero via Dynamic Circuits

T0 review · 3 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read An AlphaZero-style reinforcement-learning agent can be trained to synthesize exact unitaries under architecture constraints, and it discovers a four-T-gate Toffoli implementation that matches the proven optimal T-depth.

desk verdict Solid RL-for-synthesis paper with a real 4-T Toffoli result and an in-distribution benchmark that needs tightening before the efficiency claims land. read the letter →

arxiv 2508.21217 v1 pith:BTEXKEJH submitted 2025-08-28 quant-ph

Xavier Valcarce , Bastien Grivet , Nicolas Sangouard This is my paper · ORCID
classification quant-ph MSC 81P68 PACS 03.67.Lx
keywords unitarysynthesisAlphaZeroreinforcementlearningMonteCarlotreesearchdynamiccircuitsClifford+TgatesetcleanancillaToffoli
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper seeks to show that an AlphaZero-style reinforcement-learning agent can perform exact unitary synthesis under realistic hardware constraints: fixed discrete gate sets, limited qubit connectivity, and clean ancillas that are measured and corrected mid-circuit. It claims that after training, the agent synthesizes unitaries at low inference time, produces circuits no deeper than the random circuits that defined the targets, and recovers known optimal decompositions of standard subroutines such as Control-S, Control-T, CCZ, and Toffoli. The central specific result is a deterministic Toffoli implementation using four T gates on a clean-ancilla architecture, matching the T-depth proven optimal in earlier work that took an hour to find. The significance, if true, is practical: small subroutines invoked repeatedly inside large algorithms would become cheap to compile, potentially reducing both runtime and qubit overhead of full quantum computations.

What carries the argument

The load-bearing mechanism is a single-player game: the environment starts from an empty circuit and an input target unitary V, the state is the matrix product U_t V†, and the agent appends one allowed gate per step, with an action mask that removes commuting and redundant gates. A residual neural network with a policy head and a value head steers a Monte Carlo tree search of 200 simulations per move; the network is trained by self-play on random circuits with curriculum learning (target depth increasing), and the reward is a binary fidelity that fires only on exact match. For dynamic circuits, the same game is played on a clean ancilla: the ancilla is measured, and a second trained agent sy

What would settle it

The central 4T Toffoli claim is settled by taking the circuit in Fig. 4c and applying it to all eight computational basis states, including the branch where the ancilla measures |1> and the correction U_C is applied. If the output reproduces the Toffoli truth table with exactly four T gates, the claim holds; any deviation - a wrong output state, a missing correction, or a T-count above four - falsifies it.

Watch

Extended reading notes

Core claim

The central discovery is that a single AlphaZero-like agent, trained by self-play on random Clifford+T circuits with a fixed gate set and connectivity, can be repurposed to synthesize exact unitary implementations of standard algorithmic subroutines, including dynamic circuits that condition on a measured clean ancilla. After training on up to three data qubits plus an ancilla, the agent recovers known optimal implementations such as the 7T Toffoli on three qubits, and by pairing a candidate ancilla circuit with a separately synthesized correction unitary it finds a deterministic Toffoli implementation that uses exactly four T gates. This matches the T-depth proven optimal in Ref. [28], whic

Load-bearing premise

The benchmark and training both draw target unitaries from random circuits made from the same gate set and connectivity, so the reported success rates may not reflect how the agent performs on the structured subroutines the method is meant to compile.

Editorial extensions

If this is right

  • Small subroutines invoked millions of times inside a large algorithm can be compiled in seconds rather than hours, making the compile step a negligible part of the resource budget.
  • One trained agent works across different gate sets and qubit connectivities, so the same synthesis pipeline can target different physical processors without architectural redesign.
  • The agent's recovery of measurement-based, condition-corrected implementations suggests that dynamic circuits, not just unitary circuits, are within reach of automatic synthesis, opening a new region of the search space.
  • A deterministic Toffoli with four T gates on a clean-ancilla architecture, if replicated, lowers the non-Clifford cost of a basic building block that appears throughout quantum algorithms, potentially reducing overall fault-tolerant overhead.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • By extension, a likely untested consequence is that the same single-agent framework can be pointed at resource minimization beyond T-count - for example, measurement depth, number of ancillas, or magic-state consumption - by changing the reward function; the paper only optimizes depth and T-count.
  • One testable next step is to benchmark the trained agents on a fixed library of algorithmic primitives (two-bit additions, modular exponentiation blocks) rather than random circuits, since the random-circuit distribution used for training may overstate performance on structured inputs.
  • If the 4T Toffoli result generalizes, the combination of self-play search with post-selected correction synthesis could serve as a generic strategy to saturate proven lower bounds for ancilla-assisted circuits, where meet-in-the-middle and simulated annealing have struggled.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper presents an AlphaZero-style reinforcement-learning agent (MCTS with a ResNet policy/value network) for exact unitary synthesis over discrete gate sets on a fixed number of qubits and a given connectivity, including clean-ancilla and dynamic circuits. Targets are generated during training as random circuits drawn from the same gate set and connectivity, with a curriculum that increases circuit depth. At evaluation, the agent is benchmarked on randomly sampled targets of depth up to 40 and on a set of structured gates (CS, CT, CH, CV, iSWAP, Toffoli, CCZ, Fredkin). The agent outperforms an unguided MCTS baseline and recovers known optimal decompositions for several gates. The most striking result is a deterministic 4-T Toffoli implementation on 3 data qubits plus 1 clean ancilla, obtained from an agent-found circuit after manual deletion of seven identity gates and substitution of S for two T-dagger gates, together with a separately synthesized correction unitary. The authors claim low inference time and versatility across gate sets and connectivities, with potential impact for repeated algorithmic subroutines.

Significance. If the quantitative claims are reliable, the work extends RL-based unitary synthesis to restricted connectivity and dynamic circuits, and the 4-T Toffoli construction is an interesting resource reduction relative to static Clifford+T ancilla-free circuits. The paper provides a JAX implementation, a clear comparison to MCTS, and openly reports failures (e.g., Fredkin in the clean-ancilla architecture) and the need for human post-processing. However, the central evidence for 'versatile and efficient' rests on an in-distribution random-circuit benchmark with a weak success metric, and the non-random evaluations are a small set of well-known gates. The headline 4-T result is a human-machine co-discovery rather than a raw agent output. The significance is therefore conditional on additional structured out-of-distribution benchmarks and on automated or clearly quantified simplification.

major comments (3)
  1. [§4.2 and §5.2] The evaluation uses the same random-circuit generation process as training. Section 4.2 samples target unitaries from random circuits over the same gate set and connectivity; Section 5.2 states that benchmark targets are 'randomly sampled target unitaries' with 'the same gate set and qubit connectivity' the agent was trained on. Thus Fig. 3 measures in-distribution compression of the training distribution, not transfer to the structured algorithmic subroutines (e.g., two-bit additions for Shor's algorithm) that motivate the paper. The abstract's claim that the approach is suited for efficiently synthesizing subroutines is not quantitatively supported by this benchmark. Please add an out-of-distribution test suite of structured unitaries and compare depths/T-counts/runtimes against Synthetiq or meet-in-the-middle baselines.
  2. [§5.2, success criterion] A synthesis is called successful if the found circuit has depth at most d, where d is the depth of the random generating circuit. Since that random circuit is itself a depth-d solution, the benchmark does not measure closeness to optimality or resource efficiency; it only asks whether the agent can find a circuit no worse than a random one. The observed average synthesized depth below d is suggestive, but random circuits contain cancellations and the practically relevant comparison is against best-known or optimal depths (as partially done in Table 1). In addition, Fig. 3 reports 100 targets per depth without error bars or repeated seeds, so the 'outperformed the MCTS baseline in every case' statement lacks statistical grounding. Report confidence intervals and compare to known lower bounds or optimal depths.
  3. [§5.4, Fig. 4] The headline 4-T Toffoli result is not the raw agent output. The circuit in Fig. 4a contains seven leading gates that reduce to the identity and two T-dagger gates that are replaced by S-dagger; only after this manual simplification and the separate synthesis of a correction unitary does the 4-T implementation in Fig. 4c emerge. The paper says the improvements 'can be algorithmically addressed' but does not implement them. As written, the abstract's 'discovers unusual implementations' and the Section 5.4 wording 'this leads to a deterministic implementation... using 4T gates' attribute to the agent a circuit that required substantial human post-processing. Please quantify the manual effort, implement the claimed automated simplification, or explicitly label the result as a human-machine co-discovery.
minor comments (3)
  1. [§5.3 and Table 1] The CT gate is found only in the clean-ancilla architecture, and the Fredkin gate is not found in the clean-ancilla architecture. These negative results are disclosed, but the discussion of 'versatility' should be softened or supplemented with a more systematic study of which gates are synthesized under which architectures.
  2. [Throughout] Several typos and minor wording issues: 'ancillaes' → 'ancillas', 'outperfomed' → 'outperformed', 'resonnable' → 'reasonable', 'physicial' → 'physical', 'Tofolli' → 'Toffoli', and 'a 4−T gate implementation of the Toffoli gate, all applied on the ancillae' is unclear.
  3. [§5.4, references] The claim that the 4-T Toffoli implementation 'matches the T-depth proven optimal in Ref. [28]' cites only Ref. [28], a paper co-authored by one of the present authors. An independent reference (Jones, Ref. [14]) already appears in the bibliography and should also be cited in this context to avoid over-reliance on a self-citation.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the RL synthesis result is verified by exact fidelity; the only minor issue is a self-cited optimality comparison.

full rationale

The derivation chain is not circular. The agent's state is the matrix product U_t V^dagger and the reward is the exact binary fidelity (Eq. 1); training targets are generated by random circuits (Section 4.2) and benchmark targets are independent samples from the same generator (Section 5.2). The benchmark metric (synthesized depth <= generating depth) is a real generalization test on unseen targets, not a fitted quantity: the policy was trained on different samples, and the structured gates in Section 5.3 are external to the random training distribution. The only self-citation is the optimality comparison "This matches the T-depth proven optimal in Ref. [28]" (Section 5.4), where Ref. [28] shares an author with the present work. This is a provenance weakness, but not a circular reduction: the 4T circuit is verified by exact fidelity, and the theorem is a separate lower-bound statement. The manual simplification of Fig. 4a (dropping identity gates and replacing T-dagger T-dagger by S-dagger) is disclosed in the text as a human-in-the-loop step, so it is not presented as an automatic prediction. Overall, no equation or fitted parameter is equivalent to a claimed output by construction; the central RL result stands on its own empirical evaluation.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central claims rest on a set of RL hyperparameters (varepsilon, NMCTS, curriculum schedule, temperature) chosen by hand, and on the assumption that random circuits generated from the same gate set represent practical synthesis targets. No new physical entities are introduced. The optimality claim for the 4-T Toffoli depends on a self-cited prior result.

free parameters (5)
  • fidelity threshold epsilon = 1e-3
    Chosen to account for numerical errors in binary reward (Eq. 1); determines what counts as 'exact' synthesis.
  • MCTS simulation budget (training) = 200
    Section 4.3; chosen for resource balance.
  • MCTS simulation budget (test) = 400
    Section 5.2; chosen to balance performance and runtime for benchmark.
  • curriculum mean depth schedule = mu from 5 to 30, increment 1 per 5 epochs
    Section 4.2; defines training difficulty progression.
  • temperature schedule = T=1 first 5 steps, 0.6 for next 10, 0.4 remaining
    Section 4.2; encourages exploration then exploitation.
assumptions (4)
  • standard math Gate set {H,S,T,CNOT} is universal for the target unitaries considered
    Assumed throughout for exact synthesis on 2-3 qubits.
  • domain assumption Random circuits of depth d with the same gate set and connectivity define a representative distribution of synthesis targets
    Used in training (Section 4.2) and benchmarking (Section 5.2); no evidence that real subroutines follow this distribution.
  • domain assumption Clean-ancilla circuits with post-selection and classically-controlled corrections can implement the target unitary exactly
    Section 2; the dynamic circuit framework relies on this.
  • ad hoc to paper The T-depth optimality of the 4-T Toffoli implementation is accepted from Ref [28] without re-proof
    Section 5.4; Ref [28] shares an author with this paper, so the benchmark is self-referential.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Unitary Synthesis with AlphaZero via Dynamic Circuits." pith.science (2026). https://pith.science/paper/BTEXKEJH

@misc{pith2026250821217,
  author       = {Pith},
  title        = {Pith review of: Unitary Synthesis with AlphaZero via Dynamic Circuits},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BTEXKEJH}},
  note         = {Machine review of arXiv:2508.21217}
}
read the original abstract

Unitary synthesis is the process of decomposing a target unitary transformation into a sequence of quantum gates. This is a challenging task, as the number of possible gate combinations grows exponentially with the circuit depth. In this manuscript, we propose an approach using an AlphaZero-inspired reinforcement-learning agent for the exact compilation of unitaries using discrete sets of logic gates. The approach achieves low inference time and proves versatile across different gate sets, and qubit connectivities. Leveraging this flexibility, we explore unitary synthesis with dynamic circuits -- circuits that contain non-unitary operations such as measurements and conditional gates -- and discover unusual implementations of logical quantum gates. Although the direct synthesis of complete algorithms is intractable, our approach is well suited for efficiently synthesizing subroutines. This may have a significant impact when these subroutines are invoked repeatedly during algorithm execution.

Figures

Figures reproduced from arXiv: 2508.21217 by the authors.

Figure 1
Figure 1. Exact unitary synthesis using and AlphaZero-inspired agent. A gate set and qubit connectivity are specified. The [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Unitary synthesis success for 100 randomly sampled [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Performance of trained AlphaZero agents in synthesizing unitaries generated from random circuits of varying depth [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Using reinforcement learning to synthesize the Toffoli gate for conditionally clean ancilla architecture. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: The four steps of Monte Carlo Tree Search. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]
Figure 6
Figure 6. Figure 6: Framework of an Monte Carlo Tree Search agent. [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]
Figure 7
Figure 7. Figure 7: Framework of the training phase of AlphaZero. [PITH_FULL_IMAGE:figures/full_fig_p012_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

48 extracted references · 41 canonical work pages

  1. [20]

    Unitary Synthesis of Clifford+T Circuits with Reinforcement Learning

    Sebastian Rietsch, Abhishek Y. Dubey, Christian Ufrecht, Maniraman Periyasamy, Axel Plinge, Christopher Mutschler, and Daniel D. Scherer. “Unitary Synthesis of Clifford+T Circuits with Reinforcement Learning”. In 2024 IEEE Interna- tional Conference on Quantum Computing and Engineering (QCE). Pages 824–835. (2024)

    work page 2024

  2. [28]

    Provably optimal exact gate synthesis from a discrete gate set

    Elie Gouzien and Nicolas Sangouard. “Provably optimal exact gate synthesis from a discrete gate set” (2025). arXiv:2503.15452 [quant-ph]

    work page Pith review arXiv 2025

  3. [1]

    Quantum compiling

    Marco Maronese, Lorenzo Moro, Lorenzo Ro- cutto, and Enrico Prati. “Quantum compiling”. Page 39–74. Springer International Publishing. (2022)

    work page 2022

  4. [2]

    The solovay-kitaev algorithm

    Christopher M. Dawson and Michael A. Nielsen. “The solovay-kitaev algorithm”. Quantum Info. Comput. 6, 81–95 (2006). url: https://dl.acm .org/doi/10.5555/2011679.2011685

    arXiv 2006

  5. [3]

    Leap: Scaling numerical optimization based synthesis using an incremen- tal approach

    Ethan Smith, Marc Grau Davis, Jeffrey Larson, EdYounis, LindsayBassmanOftelie, WimLavri- jsen, and Costin Iancu. “Leap: Scaling numerical optimization based synthesis using an incremen- tal approach”. ACM Transactions on Quantum Computing4 (2023)

    work page 2023

  6. [4]

    Qfast: Conflating search and nu- merical optimization for scalable quantum circuit synthesis

    Ed Younis, Koushik Sen, Katherine Yelick, and Costin Iancu. “Qfast: Conflating search and nu- merical optimization for scalable quantum circuit synthesis”. In 2021 IEEE International Confer- ence on Quantum Computing and Engineering (QCE). Pages 232–243. (2021)

    work page 2021

  7. [5]

    Synthetiq: Fast and Versatile Quantum Circuit Synthesis

    Anouk Paradis, Jasper Dekoninck, Benjamin Bichsel, and Martin Vechev. “Synthetiq: Fast and Versatile Quantum Circuit Synthesis”. Re- production Package for the Article "Synthetiq: Fast and Versatile Quantum Circuit Synthesis" 8, 96:55–96:82 (2024)

    work page 2024

  8. [6]

    Assessing requirements to scale to practical quantum advantage

    Michael E. Beverland, Prakash Murali, Matthias Troyer, Krysta M. Svore, Torsten Hoefler, Vadym Kliuchnikov, Guang Hao Low, Math- ias Soeken, Aarthi Sundaram, and Alexan- der Vaschillo. “Assessing requirements to scale to practical quantum advantage” (2022). arXiv:2211.07629

    arXiv 2022

Show all 48 references
  1. [7]

    Practical quantum advantage in quantum simulation

    Andrew J. Daley, Immanuel Bloch, Chris- tian Kokail, Stuart Flannigan, Natalie Pearson, Matthias Troyer, and Peter Zoller. “Practical quantum advantage in quantum simulation”. Na- ture 607, 667–676 (2022)

    2022

  2. [8]

    Disentangling hype from practicality: On realistically achieving quantum advantage

    Torsten Hoefler, Thomas Häner, and Matthias Troyer. “Disentangling hype from practicality: On realistically achieving quantum advantage”. Communications of the ACM66, 82–87 (2023)

    2023

  3. [9]

    Quantum algorithms: A sur- vey of applications and end-to-end complexities

    Alexander M. Dalzell, Sam McArdle, Mario Berta, Przemyslaw Bienias, Chi-Fang Chen, An- drás Gilyén, Connor T. Hann, Michael J. Kas- toryano, Emil T. Khabiboulline, Aleksander Ku- bica, Grant Salton, Samson Wang, and Fernando G. S. L. Brandão. “Quantum algorithms: A sur- vey of...

    2025

  4. [10]

    Algorithms for quantum com- putation: discrete logarithms and factoring

    Peter W Shor. “Algorithms for quantum com- putation: discrete logarithms and factoring”. In Proceedings 35th annual symposium on foun- dations of computer science. Pages 124–134. Ieee (1994)

    1994

  5. [11]

    How to factor 2048 bit RSA integers in 8 hours using 20 million noisy qubits

    Craig Gidney and Martin Ekerå. “How to factor 2048 bit RSA integers in 8 hours using 20 million noisy qubits”. Quantum5, 433 (2021)

  6. [12]

    Factoring 2048-bit RSA Integers in 177 Days with 13 436 Qubits and a Multimode Memory

    Elie Gouzien and Nicolas Sangouard. “Factoring 2048-bit RSA Integers in 177 Days with 13 436 Qubits and a Multimode Memory”. Physical Re- view Letters127, 140503 (2021)

  7. [13]

    Quantum circuits of t-depth one

    Peter Selinger. “Quantum circuits of t-depth one”. Phys. Rev. A87, 042302 (2013)

    2013

  8. [14]

    Low-overhead constructions for the fault-tolerant Toffoli gate

    Cody Jones. “Low-overhead constructions for the fault-tolerant Toffoli gate”. Physical Review A 87, 022328 (2013)

    2013

  9. [15]

    Halving the cost of quantum ad- dition

    Craig Gidney. “Halving the cost of quantum ad- dition”. Quantum2, 74 (2018)

    2018

  10. [16]

    Surface codes: Towards practical large-scale quantum computation

    Austin G. Fowler, Matteo Mariantoni, John M. Martinis, and Andrew N. Cleland. “Surface codes: Towards practical large-scale quantum computation”. Physical Review A86 (2012)

    2012

  11. [17]

    A game of surface codes: Large- scale quantum computing with lattice surgery

    Daniel Litinski. “A game of surface codes: Large- scale quantum computing with lattice surgery”. Quantum 3, 128 (2019)

    2019

  12. [18]

    Performance analysis of a repetition cat code ar- chitecture: Computing 256-bit elliptic curve log- arithm in 9 hours with 126 133 cat qubits

    Élie Gouzien, Diego Ruiz, Francois-Marie Le Ré- gent, Jérémie Guillaud, and Nicolas Sangouard. “Performance analysis of a repetition cat code ar- chitecture: Computing 256-bit elliptic curve log- arithm in 9 hours with 126 133 cat qubits”. Phys- ical Review Letters131 (2023)

    2023

  13. [19]

    A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play

    David Silver, Thomas Hubert, Julian Schrit- twieser, Ioannis Antonoglou, Matthew Lai, ArthurGuez, MarcLanctot, LaurentSifre, Dhar- shan Kumaran, Thore Graepel, Timothy Lill- icrap, Karen Simonyan, and Demis Hassabis. “A general reinforcement learning algorithm that masters che...

    2018

  14. [21]

    A Meet-in-the-Middle 8 Algorithm for Fast Synthesis of Depth-Optimal Quantum Circuits

    Matthew Amy, Dmitri Maslov, Michele Mosca, and Martin Roetteler. “A Meet-in-the-Middle 8 Algorithm for Fast Synthesis of Depth-Optimal Quantum Circuits”. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems32, 818–830 (2013)

    2013

  15. [22]

    An algorithm for the T-count

    David Gosset, Vadym Kliuchnikov, Michele Mosca, and Vincent Russo. “An algorithm for the T-count” (2013). arXiv:1308.4134 [quant-ph]

    2013 arXiv

  16. [23]

    Paralleliz- ing quantum circuit synthesis

    Olivia Di Matteo and Michele Mosca. “Paralleliz- ing quantum circuit synthesis”. Quantum Science and Technology1, 015003 (2016)

    2016

  17. [24]

    A polynomial time and space heuristic algorithm for T-count

    Michele Mosca and Priyanka Mukhopadhyay. “A polynomial time and space heuristic algorithm for T-count”. Quantum Science and Technology 7, 015003 (2021)

    2021

  18. [25]

    A (quasi-)polynomial time heuristic algorithm for synthesizing T-depth op- timal circuits

    Vlad Gheorghiu, Michele Mosca, and Priyanka Mukhopadhyay. “A (quasi-)polynomial time heuristic algorithm for synthesizing T-depth op- timal circuits”. npj Quantum Information8, 1– 11 (2022)

    2022

  19. [26]

    Support for specific qubit connectivity

    “Support for specific qubit connectivity”.https: //github.com/eth-sri/synthetiq/issues/ 2 (2025). GitHub issue #2

    2025

  20. [27]

    Modu- lar Component-Based Quantum Circuit Synthe- sis

    Chan Gu Kang and Hakjoo Oh. “Modu- lar Component-Based Quantum Circuit Synthe- sis”. Artifact for paper "Modular Component- Based Quantum Circuit Synthesis" 7, 87:348– 87:375 (2023)

    2023

  21. [29]

    Learning high-accuracy error decoding for quantum processors

    Johannes Bausch, Andrew W. Senior, Fran- cisco J. H. Heras, Thomas Edlich, Alex Davies, Michael Newman, Cody Jones, Kevin Satzinger, Murphy Yuezhen Niu, Sam Blackwell, George Holland, DvirKafri, JuanAtalaya, CraigGidney, Demis Hassabis, Sergio Boixo, Hartmut Neven, and Pushmee...

    2024

  22. [30]

    Quantum logic gate synthesis as a Markov decision process

    M. Sohaib Alam, Noah F. Berthusen, and Pe- ter P. Orth. “Quantum logic gate synthesis as a Markov decision process”. npj Quantum Infor- mation 9, 1–10 (2023)

    2023

  23. [31]

    Variational quantum compiling with double Q-learning

    Zhimin He, Lvzhou Li, Shenggen Zheng, Yongyao Li, and Haozhen Situ. “Variational quantum compiling with double Q-learning”. New Journal of Physics23, 033002 (2021)

    2021

  24. [32]

    A Reinforcement Learning En- vironment for Directed Quantum Circuit Synthe- sis

    Michael Kölle, Tom Schubert, Philipp Altmann, Maximilian Zorn, Jonas Stein, and Claudia Linnhoff-Popien. “A Reinforcement Learning En- vironment for Directed Quantum Circuit Synthe- sis” (2024). arXiv:2401.07054 [quant-ph]

    2024 arXiv

  25. [33]

    Practical and efficient quantum circuit synthe- sis and transpiling with reinforcement learn- ing

    David Kremer, Victor Villar, Hanhee Paik, Ivan Duran, Ismael Faro, and Juan Cruz-Benito. “Practical and efficient quantum circuit synthe- sis and transpiling with reinforcement learn- ing” (2025). arXiv:2405.13196

    2025 arXiv

  26. [34]

    Simultaneous discovery of quantum error correction codes and encoders with a noise-aware reinforcement learning agent

    Jan Olle, Remmy Zen, Matteo Puviani, and Florian Marquardt. “Simultaneous discovery of quantum error correction codes and encoders with a noise-aware reinforcement learning agent”. npj Quantum Information10, 1–17 (2024)

    2024

  27. [35]

    Towards

    Marwin Segler, Mike Preuß, and Mark P. Waller. “Towards "AlphaChem": Chemical Synthesis Planning with Tree Search and Deep Neural Net- work Policies” (2017). arXiv:1702.00020 [cs]

    2017 arXiv

  28. [36]

    The Application of AlphaZero to Wargaming

    Glennn Moy and Slava Shekh. “The Application of AlphaZero to Wargaming”. In Jixue Liu and James Bailey, editors, AI 2019: Advances in Ar- tificial Intelligence. Pages 3–14. Cham (2019). Springer International Publishing

    2019

  29. [37]

    Global optimiza- tion of quantum dynamics with AlphaZero deep exploration

    Mogens Dalgaard, Felix Motzoi, Jens Jakob Sørensen, and Jacob Sherson. “Global optimiza- tion of quantum dynamics with AlphaZero deep exploration”. npj Quantum Information 6, 1– 9 (2020)

    2020

  30. [38]

    Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search

    Rémi Coulom. “Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search”. In H. Jaap van den Herik, Paolo Ciancarini, and H. H. L. M. (Jeroen) Donkers, editors, Comput- ers and Games. Pages 72–83. Berlin, Heidel- berg (2007). Springer

    2007

  31. [39]

    Quan- tum computation and quantum information: 10th anniversary edition

    Michael A. Nielsen and Isaac L. Chuang. “Quan- tum computation and quantum information: 10th anniversary edition”. Cambridge University Press. (2012)

    2012

  32. [40]

    url: http://github.com/jax-m l/jax

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang (2018). url: http://github.com/jax-m l/jax

    2018

  33. [41]

    turbozero: fast + parallel Alp- haZero

    Jacob Marshall. “turbozero: fast + parallel Alp- haZero”

  34. [42]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. “Deep residual learning for image recognition” (2015) arXiv:1512.03385

    2015 arXiv

  35. [43]

    Explorations in quantum computing

    Colin P. Williams. “Explorations in quantum computing”. Springer London. (2011)

    2011

  36. [44]

    Exact synthe- sis of multiqubit Clifford+T circuits

    Brett Giles and Peter Selinger. “Exact synthe- sis of multiqubit Clifford+T circuits”. Physical Review A87, 032332 (2013)

    2013

  37. [45]

    How can we implement controlled-t gate using CNOT and H, S and T gates?

    Craig Gidney. “How can we implement controlled-t gate using CNOT and H, S and T gates?”. url: https://quantumcomputing.sta ckexchange.com/questions/13132/how-can-w e-implement-controlled-t-gate-using-cno t-and-h-s-and-t-gates/13135#13135 . 9

  38. [46]

    Ban- dit Based Monte-Carlo Planning

    Levente Kocsis and Csaba Szepesvári. “Ban- dit Based Monte-Carlo Planning”. In Jo- hannes Fürnkranz, Tobias Scheffer, and Myra Spiliopoulou, editors, Machine Learning: ECML

  39. [47]

    Empreinte gaz à effet de serre

    EDF. “Empreinte gaz à effet de serre”. Technical report. EDF (2025). url: https://www.edf.fr /sites/groupe/files/2025-03/edf-france_ bilan-ges_2014-2024.pdf. A Monte Carlo Tree Search as agent A.1 General Monte Carlo Tree Search Algo- rithm The Monte Carlo Tree Search (MCTS) a...

    2025

  40. [2006]

    Berlin, Heidelberg (2006)

    Pages 282–293. Berlin, Heidelberg (2006). Springer

    2006

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.