Pith. sign in

REVIEW 4 major objections 6 minor 67 references

A single toy grid can teach all of reinforcement learning and then steer a qubit.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A beginner’s tutorial on classical RL with a toy quantum-control example; standard material, no new results, and multiple mathematical inconsistencies.

T0 review reviewed 2026-08-03 challenge →

load-bearing objection A promising teaching tutorial undermined by a wrong Bellman equation and arithmetic slips; not ready for students until fixed. the 4 major comments →

arxiv 2601.08662 v3 pith:ZRUVWQHO submitted 2026-01-13 cs.AI quant-ph

From Classical to Quantum Reinforcement Learning and Its Applications in Quantum Control: A Beginner's Tutorial

Abhijit Sen , Sonali Panda , Mahima Arya , Subhajit Patra , Zizhan Zheng , Denys I. Bondar This is my paper
classification cs.AI quant-ph
keywords reinforcement learningtutorialBellman equationpolicy gradientactor-criticquantum controlMarkov decision processstate-value function
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Reinforcement learning is usually taught through many scattered, math-heavy examples, but this tutorial claims that one running example—a robot on a one-dimensional grid—is enough to introduce every major RL idea in sequence: state values, Bellman equations, dynamic programming, Monte Carlo evaluation, temporal-difference learning, policy gradients, and actor-critic methods, with ready-to-run code at each step. The same example is then taken to quantum control: an actor-critic agent learns a rotation angle that drives a qubit from |0> to |1> with fidelity 0.99999998596, demonstrating that the RL machinery transfers to a quantum task without changing the framework. The point of the tutorial is pedagogical: a student who follows one example from first principles to a working quantum controller finishes with both theory and implementation skills. If the approach works, it gives undergraduates a low-cost on-ramp to both RL and quantum technology.

Core claim

On the paper's own terms, the central discovery is that a single, simple example can carry the entire RL curriculum. By reusing the same 1D grid world for every algorithm and explaining each method as a fix for what the previous one lacks, the tutorial makes the logical structure of RL visible: value iteration refines Bellman evaluation, Monte Carlo averages sampled returns, temporal-difference bootstraps, and policy-gradient methods skip value estimation altogether. The same example then serves as the testbed for an actor-critic controller that learns the optimal X-rotation for a single qubit and reaches near-unit fidelity—showing that the concepts taught on a grid apply directly to quantum

What carries the argument

The recurring 1D grid world—nine cells, a terminal cell, and a two-action policy—is the pedagogical engine: every algorithm is explained and coded in this same environment, so the reader always sees one concrete, familiar setting. The formal backbone is the Bellman expectation equation, which expresses a state's value as the immediate reward plus the discounted value of the next state; it is derived in the appendix and specialized into policy evaluation, value iteration, and the actor-critic update. The policy-gradient section adds a parameterized policy π(a|s;θ), and the quantum-control section reuses the actor-critic update with a fidelity-based reward.

Load-bearing premise

The teaching value rests on the equations being accurate as written; the Bellman equation in Eq. (23) sums only over s′≠s, an omission that is false in the tutorial's own stochastic grid where moving 'right' from s1 can keep the agent in s1 with probability 0.3.

What would settle it

In the tutorial's stochastic grid from Section VII, compute the value of state s1 using Eq. (23) as written (excluding s′=s1) and compare it with the value obtained by including the self-loop p(s1|s1,'right')=0.3 in the Bellman sum; the mismatch shows the equation needs correction for environments with self-transitions.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • A student who follows the tutorial should be able to implement value iteration, Q-learning, SARSA, and actor-critic on a small grid, because each is presented as running code in the same environment.
  • The quantum-control example shows that RL control does not require knowing the system Hamiltonian analytically—only a state encoding, actions, and a fidelity reward—so the same code style transfers to other quantum tasks.
  • The 'what is missing in the previous method' presentation gives an instructor a ready-made ordering for a short RL course, reducing the time students spend reconciling unrelated examples.
  • The appendix derivations of the Bellman equation and first-visit/every-visit Monte Carlo let a determined student trace every algorithm back to first principles without leaving the tutorial.
  • The tracking-control discussion suggests the same RL setup can be pointed at time-dependent objectives (minimizing accumulated tracking error) rather than terminal fidelity, which is a natural next step for quantum control.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The single-example strategy could be stress-tested by asking students to transfer to a second environment (e.g., a 2D grid or a continuous-control benchmark); transfer success would strengthen the pedagogical claim beyond the tutorial's own scope.
  • The reported qubit fidelity is for a toy single-qubit rotation; extending the same actor-critic framework to multi-qubit state preparation or to control under noise would show whether the quantum-control transfer is more than illustrative.
  • The paper's tracking-control sketch opens a concrete benchmark: train a policy on a quadratic tracking-error reward and compare its trajectory-following against the analytical inversion controller in a noisy environment. This would test whether RL genuinely replaces the explicit inversion step.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper is a tutorial aimed at undergraduate students, with the stated goal of making reinforcement learning accessible by explaining all core concepts through a single simple 1D grid example, providing ready-to-use Python code, and then extending the discussion to quantum control and quantum reinforcement learning. It covers probability preliminaries, policies, value functions, Bellman equations, dynamic programming, Monte Carlo methods, temporal-difference learning, policy gradient, actor-critic methods, and an application to qubit rotation control.

Significance. If the exposition were mathematically reliable, this would be a genuinely useful teaching supplement: it is example-driven, self-contained, connects theory to code, and explicitly targets undergraduates. The GitHub code availability and the appendices with derivations are strengths, and the single-example structure is pedagogically appealing. However, several internal errors occur in exactly the equations and worked examples that carry the teaching load, so the tutorial in its current form risks misleading the intended audience.

major comments (4)
  1. [Sec. VII, Eq. (23); Appendix A] The Bellman expectation equation is written with the inner sum over s' in S\{s}, and the text makes this restriction explicit ('for all s in S, s' in S\{s}'). This is not the standard Bellman equation when self-transitions exist. The paper itself introduces such self-transitions in Sec. VII (e.g., p(s1|s1,right)=0.3) and in the original 1D table (s1 left, s9 right). Omitting s'=s discards p(s,r|s,a)[r+gamma v_pi(s)] and changes the solution of the linear system. Appendix A derives the standard full sum over all s' (Eq. A7) and states it corresponds to Eq. (23), so the paper is internally inconsistent. Eq. (32) repeats the erroneous restriction and also contains a typo ('s' in S\{s3}'). A student implementing Eq. (23) on the paper's own stochastic grid will compute wrong state values. Please replace the sum over S\{s} with S and correct Eq. (32) and the surrounding text.
  2. [Sec. V, action-value example] The worked action-value example is inconsistent. The trajectory starting from s2 with action left contains five -1 rewards and one +5 reward; their sum is 0, not 1 as stated. The text then concludes q_pi(s1,left)=1, although the episode begins in s2 (Fig. 3). Please correct both the return and the state label. Since this is the only fully worked q_pi example, it is load-bearing for the tutorial's presentation of action-value functions.
  3. [Sec. X, Monte Carlo Methods] The average over the two returns for state s5 is computed as (2+2)/2 = 1. The correct result is 2. This arithmetic error occurs in the central demonstration of Monte Carlo averaging and must be fixed. Please also verify that the returns assigned to the red and blue trajectories match the paths in Fig. 5.
  4. [Sec. IX, Dynamic Programming] For state s6, the text states that the existing 'right' policy gives a state-value of -3 and that choosing 'left' gives an action-value of +3. No derivation is given, and these values do not follow from the 1D example referenced in that section (reward -1 per step, +5 at terminal; the original 'right' policy from s6 eventually self-loops at s9). If a different reward/truncation scheme is intended, this must be stated explicitly; otherwise the values should be recomputed. The section also mixes reward constants +5 and +3 without signposting, which is confusing in a tutorial.
minor comments (6)
  1. [Sec. VIII] The text writes p(s',r|s1,a); this should be p(s',r|s,a).
  2. [Sec. XIII, Eqs. (49)-(50)] The time index in the temporal-difference error is written as r_t, whereas the earlier notation uses r_{t+1}. Please align the indexing.
  3. [Sec. XII, parameterized policy table] For state s1, the table gives pi(left)=0 and pi(right)=1 - theta_1, which sums to 1 only when theta_1=0. Please clarify that theta_1 is frozen or define pi(left)=theta_1, pi(right)=1-theta_1 so that the policy is a valid probability distribution for all allowed parameter values.
  4. [References [46]] Reference [46] is an incomplete placeholder ('A. Author and B. Collaborators'). It must be completed with a verifiable citation or removed.
  5. [Fig. 2] The reward row appears to place +5 under s4 rather than under the terminal state s5, and the list appears to contain only eight reward entries for nine states. Please correct the figure for consistency with the text.
  6. [General] Reward constants change between +5 and +3 across sections without explicit signposting. Adding a short summary of the reward scheme for each example would prevent confusion.

Circularity Check

0 steps flagged

No circular derivation: standard RL identities derived from definitions; only a minor non-load-bearing self-citation.

full rationale

The paper's derivations are standard textbook material and are not circular. The Bellman expectation equation (Eq. 23, Appendix A) is obtained from the return recursion Gt = rt+1 + γGt+1 and the definition vπ(s) = Eπ[Gt | St=s]; these are definitions, not assumed conclusions. The policy-evaluation, TD, REINFORCE, and actor-critic updates are standard identities cited to external textbooks [15,19,20,26], and the quantum-control example is a simulation whose fidelity is an output, not a fitted input. The QRL section explicitly states that 'the loop is exactly the classical policy-gradient algorithm; the only change is that the function class for π is a quantum circuit' (Sec. XVI.C), so no renamed result or imported uniqueness is used. The only self-reference is the prerequisite pointer to the authors' own GitHub booklet [7] in the Introduction ('It is advised to refer to the following resource as a prerequisite [7]'), which is ancillary rather than load-bearing; it does not substitute for any derivation. No other self-citation chain appears. The internal inconsistency between Eq. (23)/Eq. (32), which exclude self-transitions, and Appendix A, which sums over all next states, together with the stochastic transition table with self-loops in Sec. VII, is a genuine mathematical/correctness flaw but not a circular reduction; similarly, placeholder ref. [46] is a bibliographic defect, not circularity. Consistent with the scale, the presence of one minor non-load-bearing self-citation warrants score 2 rather than 0.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 1 invented entities

The paper is a tutorial and introduces no new physical entities or fitted science. Its worked numbers depend on hand-chosen hyperparameters (α, γ, rewards) and on standard RL/physics assumptions. The main formal issue is Eq. (23), which improperly drops self-transitions.

free parameters (5)
  • α (learning rate) = 0.1
    Hand-chosen in TD, REINFORCE, and actor-critic worked examples; controls update sizes but is not fitted to data.
  • γ (discount factor) = 0.9 (γ=1 in some sections)
    Hand-chosen discount in numerical examples; affects numeric state values.
  • reward constants = -1 per step, +5 at terminal
    Arbitrary reward values selected to make the grid example easy; not derived from a problem.
  • policy parameter θ = initialized to 0; one update gives θ6=0.09, θ7 clipped to -0.5
    Trainable parameter in policy-gradient and actor-critic illustrations; chosen for hand calculation, not learned from data.
  • qubit control angle θ = ~π (fidelity 0.99999998596)
    The actor-critic example learns a rotation angle for a single-qubit X-gate; the near-unit fidelity is the outcome of a toy problem, not a measurement.
axioms (6)
  • domain assumption Markov property: next state and reward depend only on the current state and action
    Stated in Sec. VIII to justify the MDP formulation; standard but not true of every environment.
  • domain assumption Scalar reward hypothesis: all goals can be expressed as cumulative scalar rewards
    Used throughout; e.g., the grid uses -1/+5 and quantum control uses fidelity as reward.
  • domain assumption Quantum postulates: superposition, measurement collapse, unitary evolution, entanglement, no-cloning, interference
    Listed in Sec. XVI as the rules of quantum computing; standard but not proved.
  • domain assumption Time-dependent Schrödinger equation governs control dynamics
    Eq. (52) in Sec. XV is the basis of the quantum-control formulation.
  • standard math Convergence of iterative Bellman, policy, and value iteration
    Appendices D/E rely on limits converging to vπ and v*; standard result from Sutton & Barto, asserted without proof.
  • standard math Probability axioms, conditional expectation, and linearity of expectation
    Used in Sec. III and Appendix A to derive the Bellman equation.
invented entities (1)
  • 'mysterious force' in the environment no independent evidence
    purpose: Pedagogical explanation for stochastic transition probabilities
    Sec. IV attributes stochasticity to a 'mysterious force'; this is a metaphor, not a physical mechanism.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of From Classical to Quantum Reinforcement Learning and Its Applications in Quantum Control: A Beginner's Tutorial." pith.science (2026). https://pith.science/paper/ZRUVWQHO

@misc{pith2026260108662,
  author       = {Pith},
  title        = {Pith review of: From Classical to Quantum Reinforcement Learning and Its Applications in Quantum Control: A Beginner's Tutorial},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZRUVWQHO}},
  note         = {Machine review of arXiv:2601.08662}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

This tutorial is designed to make reinforcement learning (RL) more accessible to undergraduate students by offering clear, example-driven explanations. It focuses on bridging the gap between RL theory and practical coding applications, addressing common challenges that students face when transitioning from conceptual understanding to implementation. Through hands-on examples and approachable explanations, the tutorial aims to equip students with the foundational skills needed to confidently apply RL techniques in real-world scenarios.

Figures

Figures reproduced from arXiv: 2601.08662 by Abhijit Sen, Denys I. Bondar, Mahima Arya, Sonali Panda, Subhajit Patra, Zizhan Zheng.

Figure 1
Figure 1. Figure 1: FIG. 1: A 1D grid with a robot in cell [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: FIG. 2: An improved version of Figure [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: FIG. 3: Another version of Fig [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: FIG. 5: Robot’s trajectories from cell s3 to the target [PITH_FULL_IMAGE:figures/full_fig_p010_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: FIG. 6: Grid world with robot in [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: FIG. 7: Improved policy with action directions and [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: FIG. 8: Bloch sphere representations of quantum state evolution. ( [PITH_FULL_IMAGE:figures/full_fig_p018_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

67 extracted references · 5 linked inside Pith

  1. [1]

    We can, in this case, calculate state-action valueqπ(s,a)

    Given the agent is in states, choose a new action athat the agent takes in current state and then follows policyπ. We can, in this case, calculate state-action valueqπ(s,a)

  2. [2]

    5: Robot’s trajectories from cell s3 to the target cell s7 with different paths represented by colored arrows

    If qπ(s,a)>v π(s), s1 s2 s3 s4 s5 s6 s7 s8 s9 FIG. 5: Robot’s trajectories from cell s3 to the target cell s7 with different paths represented by colored arrows. it suggests that taking actionaand then following the current policy is better than just following the policyπall the time from that state

  3. [3]

    right", resulting in a state-value of−3. How- ever, if we were to assign a new action

    Update the policy to chooseains. Let us revisit our one-dimensional example illustrated in Figure 2, where the policy and transitions are detailed in Sec. IV. For states6, the existing policy prescribes the action"right", resulting in a state-value of−3. How- ever, if we were to assign a new action"left"for state s6, we could transition to the terminal st...

    1988

  4. [4]

    Master- ing atari, go, chess and shogi by planning with a learned model.Nature, 588(7839):604–609, December 2020

    Julian Schrittwieser, Ioannis Antonoglou, Thomas Hu- bert, Karen Simonyan, Laurent Sifre, Simon Schmitt, Arthur Guez, Edward Lockhart, Demis Hassabis, Thore Graepel, Timothy Lillicrap, and David Silver. Master- ing atari, go, chess and shogi by planning with a learned model.Nature, 588(7839):604–609, December 2020

    2020

  5. [5]

    A general reinforcement learning algorithm that masters chess, shogi, and go through self-play.Science, 362(6419):1140–1144, December 2018

    David Silver, Thomas Hubert, Julian Schrittwieser, Ioan- nis Antonoglou, Matthew Lai, Arthur Guez, Marc Lanc- tot, Laurent Sifre, Dharshan Kumaran, Thore Graepel, Timothy Lillicrap, Karen Simonyan, and Demis Hass- abis. A general reinforcement learning algorithm that masters chess, shogi, and go through self-play.Science, 362(6419):1140–1144, December 2018

    2018

  6. [6]

    Bellemare, Salvatore Candido, Pablo Samuel Castro, Jun Gong, Marlos C

    Marc G. Bellemare, Salvatore Candido, Pablo Samuel Castro, Jun Gong, Marlos C. Machado, Subhodeep Moitra, Sameera S. Ponda, and Ziyu Wang. Autonomous navigation of stratospheric balloons using reinforcement learning.Nature, 588(7836):77–82, December 2020

    2020

  7. [7]

    A review on reinforcement learning-based highway autonomous ve- hicle control.Green Energy and Intelligent Transporta- tion, 3(4):100156, August 2024

    Ali Irshayyid, Jun Chen, and Guojiang Xiong. A review on reinforcement learning-based highway autonomous ve- hicle control.Green Energy and Intelligent Transporta- tion, 3(4):100156, August 2024

    2024

  8. [8]

    Al-Masrur Khan, Md Rashed Jaowad Khan, Abul Tooshil, Niloy Sikder, M

    Md. Al-Masrur Khan, Md Rashed Jaowad Khan, Abul Tooshil, Niloy Sikder, M. A. Parvez Mahmud, Abbas Z. Kouzani, and Abdullah-Al Nahid. A systematic review on reinforcement learning-based robotics within the last decade.IEEE Access, 8:176598–176623, 2020

    2020

  9. [9]

    Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Car- roll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, JanLeike, andRyanLowe. Traininglanguage models to follow instructions with human feedback. I...

    2022

  10. [10]

    Reinforcement tu- torial for undergraduates — booklets.https: //github.com/asen009/Reinforcement_Tutorial_ Undergraduates/tree/main/Booklets, 2025

    Abhijit Sen Sonali Panda. Reinforcement tu- torial for undergraduates — booklets.https: //github.com/asen009/Reinforcement_Tutorial_ Undergraduates/tree/main/Booklets, 2025. GitHub repository

    2025

  11. [11]

    An invitation to deep reinforcement learning, 2023

    Bernhard Jaeger and Andreas Geiger. An invitation to deep reinforcement learning, 2023

    2023

  12. [12]

    Reinforcement learning: A tutorial sur- vey and recent advances.INFORMS Journal on Com- puting, 21(2):178–192, 2009

    Abhijit Gosavi. Reinforcement learning: A tutorial sur- vey and recent advances.INFORMS Journal on Com- puting, 21(2):178–192, 2009

    2009

  13. [13]

    Reinforcement learning: A friendly introduction, 2021

    Verena Heidrich-Meisner, Martin Lauer, Christian Igel, and Martin Riedmiller. Reinforcement learning: A friendly introduction, 2021. Accessed: 2025-05-06

    2021

  14. [14]

    Stephanie S. Harmon. Reinforcement learning: A tuto- rial, 2000. Accessed: 2025-05-06

    2000

  15. [15]

    Application of reinforcement learning and deep learning in multiple- input and multiple-output (mimo) systems.Sensors, 22(1):309, 2022

    Giuseppe De Pietro and Antonio Coronato. Application of reinforcement learning and deep learning in multiple- input and multiple-output (mimo) systems.Sensors, 22(1):309, 2022. 22

    2022

  16. [16]

    html#dictionariesfor details about Python dictionar- ies

    Please consult the official documentationhttps: //docs.python.org/3/tutorial/datastructures. html#dictionariesfor details about Python dictionar- ies

  17. [17]

    Note there is a separate data type in Python for ordered dictionarieshttps://docs.python.org/3/ library/collections.html#ordereddict-objects

  18. [18]

    Adaptive Computation and Machine Learn- ing series

    Richard S Sutton and Andrew G Barto.Reinforcement Learning. Adaptive Computation and Machine Learn- ing series. Bradford Books, Cambridge, MA, 2 edition, November 2018. [16]https://docs.python.org/3/tutorial/ datastructures.html#more-on-lists. [17]https://docs.python.org/3/tutorial/ datastructures.html#tuples-and-sequences

    2018

  19. [19]

    An introduction to reinforcement learning: Fundamental concepts and practical applications, 2024

    Majid Ghasemi, Amir Hossein Moosavi, Ibrahim Sorkhoh, Anjali Agrawal, Fadi Alzhouri, and Dariush Ebrahimi. An introduction to reinforcement learning: Fundamental concepts and practical applications, 2024

    2024

  20. [20]

    Richard S. Sutton. Learning to predict by the methods of temporal differences.Machine Learning, 3(1):9–44, August 1988

    1988

  21. [21]

    Williams

    Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning.Ma- chine Learning, 8(3-4):229–256, 1992

    1992

  22. [22]

    Policy gradient methods in rein- forcement learning, 2020

    Rishabh Khandelwal. Policy gradient methods in rein- forcement learning, 2020. Accessed: 2025-08-12

    2020

  23. [23]

    Three fundamental flaws in common re- inforcement learning algorithms (and how to fix them),

    Lilian Weng. Three fundamental flaws in common re- inforcement learning algorithms (and how to fix them),

  24. [24]

    Du, Lin F

    Ruosong Wang, Simon S. Du, Lin F. Yang, and Sham M. Kakade. Is long horizon reinforcement learning more dif- ficult than short horizon reinforcement learning? In Advances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  25. [25]

    Natural actor-critic

    Jan Peters and Stefan Schaal. Natural actor-critic. In Proceedings of the 25th International Conference on Ma- chine Learning, pages 768–775. ACM, 2008

    2008

  26. [26]

    Barto, Richard S

    Andrew G. Barto, Richard S. Sutton, and Charles W. Anderson. Neuronlike adaptive elements that can solve difficult learning control problems.IEEE Transactions on Systems, Man, and Cybernetics, SMC-13(5):834–846, 1983

    1983

  27. [27]

    Sample efficient reinforcement learning withREINFORCE

    Junzi Zhang, Jongho Kim, Brendan O’Donoghue, and Stephen Boyd. Sample efficient reinforcement learning withREINFORCE. InProceedings of the AAAI Confer- ence on Artificial Intelligence, volume 35, pages 10887– 10895, 2021

    2021

  28. [28]

    D. J. Tannor and S. A. Rice. Control of selectivity of chemical reaction via control of wave packet evolu- tion.The Journal of Chemical Physics, 83(10):5013– 5018, 1985

    1985

  29. [29]

    Control of unimolecu- lar reactions using coherent light.Chemical Physics Let- ters, 126(6):541–546, 1986

    Paul Brumer and Moshe Shapiro. Control of unimolecu- lar reactions using coherent light.Chemical Physics Let- ters, 126(6):541–546, 1986

    1986

  30. [30]

    Assion, T

    A. Assion, T. Baumert, M. Bergt, T. Brixner, B. Kiefer, V. Seyfried, M. Strehle, and G. Gerber. Control of chem- ical reactions by feedback-optimized phase-shaped fem- tosecond laser pulses.Science, 282(5390):919–922, 1998

    1998

  31. [31]

    W. S. Warren, H. Rabitz, and M. Dahleh. Coherent control of quantum dynamics.Science, 259(5101):1581– 1589, 1993

    1993

  32. [32]

    A. P. Peirce, M. A. Dahleh, and H. Rabitz. Optimal con- trol of quantum-mechanical systems: Existence, numeri- cal approximation, and applications.Physical Review A, 37(12):4950–4964, 1988

    1988

  33. [33]

    J. L. Herek, W. Wohlleben, R. J. Cogdell, D. Zeidler, and M. Motzkus. Quantum control of energy flow in light-harvesting complexes.Nature, 417:533–535, 2002

    2002

  34. [34]

    Marcel Dekker, New York, 1996

    V.F.Krotov.Global Methods in Optimal Control Theory. Marcel Dekker, New York, 1996

    1996

  35. [35]

    Khaneja, T

    N. Khaneja, T. Reiss, C. Kehlet, T. Schulte-Herbrüggen, and S. J. Glaser. Optimal control of coupled spin dynam- ics: Design of nmr pulse sequences by gradient ascent algorithms.Journal of Magnetic Resonance, 172(2):296– 305, 2005

    2005

  36. [36]

    D. J. Egger and F. K. Wilhelm. Adaptive hybrid optimal quantum control for imprecisely characterized systems. Physical Review Letters, 112(24):240503, 2014

    2014

  37. [37]

    Caneva, T

    T. Caneva, T. Calarco, and S. Montangero. Chopped random-basis quantum optimization.Physical Review A, 84(2):022326, 2011

    2011

  38. [38]

    S. J. Glaser, U. Boscain, T. Calarco, C. P. Koch, W. Köckenberger, R. Kosloff, I. Kuprov, B. Luy, S. Schirmer, T. Schulte-Herbrüggen, D. Sugny, and F. K. Wilhelm. Training schrödinger’s cat: Quantum optimal control. strategic report on current status, visions and goalsforresearchineurope.The European Physical Jour- nal D, 69(12):279, 2015

    2015

  39. [39]

    Machnes, U

    S. Machnes, U. Sander, S. J. Glaser, P. de Fouquieres, A. Gruslys, S. Schirmer, and T. Schulte-Herbrüggen. Comparing, optimizing, and benchmarking quantum- controlalgorithmsinaunifyingprogrammingframework. Physical Review A, 84(2):022305, 2011

    2011

  40. [40]

    Bukov, A

    D. Bukov, A. G. R. Day, P. Weinberg, A. Polkovnikov, and P. Mehta. Reinforcement learning in different phases of quantum control.Physical Review X, 8:031086, 2018

    2018

  41. [41]

    Ansel, R

    A. Ansel, R. C. B. Lucherini, M. Reagor, M. J. Gullans, M. Devoret, and L. I. Glazman. Machine learning opti- mization of quantum control in superconducting qubits. npj Quantum Information, 7(94), 2021

    2021

  42. [42]

    Skolik, J

    A. Skolik, J. R. McClean, M. Mohseni, P. van der Smagt, and V. Dunjko. Reinforcement learning for quantum con- trol and quantum error correction.Quantum Science and Technology, 7(1):015002, 2022

    2022

  43. [43]

    Zhang, M

    Z. Zhang, M. Sarovar, and K. B. Whaley. Deep reinforce- ment learning for quantum gate control.Physical Review Letters, 122(2):020501, 2019

    2019

  44. [44]

    Koutromanos, D

    D. Koutromanos, D. Stefanatos, and E. Paspalakis. Con- trol of qubit dynamics using reinforcement learning.In- formation, 15(5):272, 2024

    2024

  45. [45]

    Y. Gao, X. Wang, N. Yu, and B. M. Wong. Harnessing deep reinforcement learning to construct time-dependent optimal fields for quantum control dynamics.Physical Chemistry Chemical Physics, 24(39):24208–24217, 2022

    2022

  46. [46]

    Robustquantumcontrol using reinforcement learning.npj Quantum Information,

    A.AuthorandB.Collaborators. Robustquantumcontrol using reinforcement learning.npj Quantum Information,

  47. [47]

    J. O. Ernst, A. Chatterjee, T. Franzmeyer, and A. Kuhn. Reinforcement learning for quantum control under phys- ical constraints.arXiv preprint arXiv:2501.14372, 2025

    Pith/arXiv arXiv 2025

  48. [48]

    Smelyan- skiy, and Hartmut Neven

    Murphy Yuezhen Niu, Sergio Boixo, Vadim N. Smelyan- skiy, and Hartmut Neven. Universal quantum control through deep reinforcement learning.npj Quantum In- 23 formation, 5(1), April 2019

    2019

  49. [49]

    Deep reinforcement learning for quantum state preparation with weak nonlinear measure- ments.Quantum, 6:747, 2022

    Riccardo Porotti, Antoine Essig, Benjamin Huard, and Florian Marquardt. Deep reinforcement learning for quantum state preparation with weak nonlinear measure- ments.Quantum, 6:747, 2022

    2022

  50. [50]

    Shuang and H

    F. Shuang and H. Rabitz. Control of quantum observ- ables by tracking control fields.Journal of Chemical Physics, 121:9270–9278, 2004

    2004

  51. [51]

    Reinforcement learning with neural networks for quantum feedback.Quantum, 6:747, 2022

    Thomas Fösel, Petru Tighineanu, Thomas Weiss, and Florian Marquardt. Reinforcement learning with neural networks for quantum feedback.Quantum, 6:747, 2022

    2022

  52. [52]

    Springer, 1995

    Alberto Isidori.Nonlinear Control Systems. Springer, 1995

    1995

  53. [53]

    Dong and I.R

    D. Dong and I.R. Petersen. Quantum control theory and applications: a survey.IET Control Theory & Applica- tions, 4(12):2651–2671, December 2010

    2010

  54. [54]

    Williams

    Ronald J. Williams. Simple statistical gradient-following algorithms for connectionist reinforcement learning.Ma- chine Learning, 8(3):229–256, May 1992

    1992

  55. [55]

    Marino and P

    R. Marino and P. Tomei. Nonlinear control design: Ge- ometric, adaptive and robust.Prentice Hall, 1995

    1995

  56. [56]

    Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.arXiv preprint arXiv:1707.06347, 2017

    Pith/arXiv arXiv 2017

  57. [57]

    Jordan, and Pieter Abbeel

    John Schulman, Sergey Levine, Philipp Moritz, Michael I. Jordan, and Pieter Abbeel. Trust region pol- icyoptimization. InProceedings of the 32nd International Conference on Machine Learning (ICML), volume 37 of Proceedings of Machine Learning Research, pages 1889–

  58. [58]

    Addressing function approximation error in actor-critic methods.arXiv preprint arXiv:1802.09477, 2018

    Scott Fujimoto, Herke van Hoof, and David Meger. Addressing function approximation error in actor-critic methods.arXiv preprint arXiv:1802.09477, 2018

    Pith/arXiv arXiv 2018

  59. [59]

    Lillicrap, Jonathan J

    Timothy P. Lillicrap, Jonathan J. Hunt, Alexander Pritzel, Nicolas Heess, Tom Erez, Yuval Tassa, David Sil- ver, and Daan Wierstra. Continuous control with deep reinforcementlearning.arXiv preprint arXiv:1509.02971, 2015

    Pith/arXiv arXiv 2015

  60. [60]

    Reinforcement learning: Computational theory and biological mechanisms.HFSP Journal, 1(1):30–40, May 2007

    Kenji Doya. Reinforcement learning: Computational theory and biological mechanisms.HFSP Journal, 1(1):30–40, May 2007

    2007

  61. [61]

    Soft actor-critic algorithms and appli- cations.arXiv preprint arXiv:1812.05905, 2018

    Tuomas Haarnoja, Aurick Zhou, Pieter Abbeel, and Sergey Levine. Soft actor-critic algorithms and appli- cations.arXiv preprint arXiv:1812.05905, 2018

    Pith/arXiv arXiv 2018

  62. [62]

    Quantum framework for reinforcement learning: Integrating the markov decision process, quantum arith- metic, and trajectory search.Physical Review A, 111(6), June 2025

    Thet Htar Su, Shaswot Shresthamali, and Masaaki Kondo. Quantum framework for reinforcement learning: Integrating the markov decision process, quantum arith- metic, and trajectory search.Physical Review A, 111(6), June 2025

    2025

  63. [63]

    An introduction to quantum re- inforcement learning (qrl), 2024

    Samuel Yen-Chi Chen. An introduction to quantum re- inforcement learning (qrl), 2024

    2024

  64. [64]

    Hands-on introduction to quantum machine learning.The International FLAIRS Conference Proceed- ings, 37, May 2024

    Muhammad Ismail, Mohamed Shaban, and Samuel Yen- Chi Chen. Hands-on introduction to quantum machine learning.The International FLAIRS Conference Proceed- ings, 37, May 2024

    2024

  65. [65]

    Quantum reinforcement learning, 2023

    Shao hua Wang and Hai Lin. Quantum reinforcement learning, 2023

    2023

  66. [67]

    Springer Interna- tional Publishing, 2021

    Maria Schuld and Francesco Petruccione.Machine Learning with Quantum Computers. Springer Interna- tional Publishing, 2021. Appendix A: Bellman Equation Derivation The Bellman expectation equation, introduced in Sec. VII [Eq. 23], plays a central role in the analysis presented in this work. For completeness, its derivation is outlined in this appendix, sta...

    2021

  67. [2019]

    Accessed: 2025-08-12

    2025

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.