Pith. sign in

REVIEW 4 major objections 4 minor 40 references

Accelerating Quantum Reinforcement Learning with a Quantum Natural Policy Gradient Based Approach

T0 review · 4 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a quantum natural policy gradient algorithm with deterministic truncated estimators and quantum mean estimation achieves $\tilde{\mathcal{O}}(\epsilon^{-1.5})$ sample complexity for model-free infinite-horizon…

desk verdict Serious, plausible result with a fixable oracle-model gap: the stated quantum access model has no reward oracle, but the gradient estimator needs reward values. read the letter →

arxiv 2501.16243 v3 pith:WQ4FZLMH submitted 2025-01-27 quant-ph cs.AIstat.ML

Yang Xu , Vaneet Aggarwal This is my paper · ORCID
classification quant-phcs.AIstat.ML MSC 68Q1290C4068T0581P68
keywords quantumreinforcementlearningnaturalpolicygradientmeanestimationsamplecomplexityMarkovdecisionprocessmodel-freeoracle
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper claims that quantum access to an MDP's transition and initial-state distributions can accelerate model-free policy gradient reinforcement learning below the classical sample-complexity floor. It introduces the Quantum Natural Policy Gradient (QNPG) algorithm, which replaces random geometric-length trajectory sampling with fixed-length truncated trajectories that can be generated coherently in superposition. Truncation introduces exponentially decaying bias, but quantum mean estimation reduces the variance of the resulting gradient and Fisher information estimates fast enough to yield $\tilde{\mathcal{O}}(\epsilon^{-1.5})$ oracle queries for an $\epsilon$-optimal policy. If the theorem holds, this is the first quantum speedup for general-parameterization model-free infinite-horizon RL, with the caveat that the algorithm assumes rewards are classically computable from state-action pairs.

What carries the argument

The load-bearing machinery is a coherent quantum embedding of the truncated NPG estimator: a unitary $U_P(\tau_N)$ prepares a superposition over all length-$N$ trajectories using the initial-state oracle and $O(N)$ calls to the transition and policy oracles, then unitaries $U_F$ and $U_g$ append the Fisher matrix estimate and gradient estimate in superposition. These are combined with QVarianceReduce, a quantum mean-estimation-based variance reduction routine that converts an estimator with variance $L^2$ into one with variance $\hat{\sigma}^2$ using $\tilde{O}(L\sqrt{d}/\hat{\sigma})$ queries. The deterministic truncation in Eqs. (14)-(15) is what makes the whole pipeline implementable in superposition; the resulting bias decays as $O(\gamma^N)$, which the analysis absorbs by choosing $N=O(\log(1/\epsilon))$.

What would settle it

Check whether a reward unitary is available: Section 2.3 defines only $U_P$, $U_\rho$, and $\Pi$, while Eq. (14) requires $r(s_n,a_n)$ and Appendix A.2 assumes it is classically computable. If the reward is accessible only through a black-box oracle, then $U_g$ in Eq. (17) cannot be built from the stated oracles, and Theorem 3's query count omits the reward-query cost; alternatively, simulate Theorem 1's bias bounds (26)-(29) on a small MDP and test whether the truncation bias decays as $O(\gamma^N)$.

Watch

Extended reading notes

Core claim

The central claim is Theorem 3: under Lipschitz and smooth score functions, a bounded compatible function approximation error $\epsilon_{\mathrm{bias}}$, and Fisher non-degeneracy, Algorithm 1 with $H=O(\log(1/\epsilon))$, $N=O(\log(1/\epsilon))$, $K=O(1/\epsilon)$, $\hat{\sigma}_g^2=O(\epsilon)$, and $\hat{\sigma}_F^2=O((1-\gamma)^4\epsilon)$ guarantees $J^*_\rho - (1/K)\sum_{k=0}^{K-1} E[J_\rho(\theta_k)] \le \sqrt{\epsilon_{\mathrm{bias}}} + \epsilon$ using $\tilde{\mathcal{O}}(\epsilon^{-1.5})$ quantum oracle queries and $\tilde{\mathcal{O}}(\epsilon^{-1})$ iterations. The speedup comes from replacing classical geometric-length trajectory sampling with deterministic length-$N$ truncation and feeding the resulting estimators into quantum mean estimation, which reduces variance quadratically faster than classical averaging. The authors state this is the first demonstration of a quantum speedup for parameterized model-free infinite-horizon MDPs.

Load-bearing premise

The guarantee assumes the reward of any state-action pair is known classically to the learner, so it can be encoded coherently into the gradient and Fisher estimators; the quantum model stated in the paper alone, with only transition and initial-state oracles, does not provide reward access.

Editorial extensions

If this is right

  • Model-free policy gradient RL with general policy parameterization can be solved with $\tilde{\mathcal{O}}(\epsilon^{-1.5})$ quantum oracle queries, improving on the classical $\tilde{\mathcal{O}}(\epsilon^{-2})$ lower bound for MDP queries.
  • The outer loop still needs $\tilde{\mathcal{O}}(\epsilon^{-1})$ iterations; the speedup lives entirely in the inner-loop estimation, making QNPG a hybrid classical-outer, quantum-inner algorithm.
  • A trajectory length of only $N=O(\log(1/\epsilon))$ is sufficient, because the truncation bias decays exponentially in $N$ with rate $\gamma$.
  • Under softmax parameterization, where $\epsilon_{\mathrm{bias}} = 0$, the guarantee becomes a clean $\epsilon$-optimal value gap; under expressive neural parameterization the gap inherits the compatible approximation error $\epsilon_{\mathrm{bias}}$.
  • The query count is measured in calls to $U_\rho$, $U_P$, and $\Pi$, and Lemma 4 accounts for the $O(N)$ transitions and policy applications inside each superposition sample.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • In my reading, the reward-access assumption is the soft point: the oracle model in Section 2.3 defines only transition and initial-state oracles, yet Eq. (14) evaluates $r(s_n,a_n)$, and Appendix A.2 fills the gap by assuming rewards, score functions, and discounting are classically computable in superposition. If a future model-free setting hides rewards behind a quantum oracle, each $U_g$ call m
  • The speedup is a polynomial $\epsilon^{1/2}$ factor, not exponential; because the outer loop remains classical, further improvement would require a different outer-loop mechanism such as quantum cutting-plane methods, which the paper argues cannot be directly adapted to non-convex RL.
  • A practical extension is to tune $N$ against the actual bias: the analysis guarantees $N=O(\log(1/\epsilon))$, but constants hidden in the $\tilde{O}$ bounds are not quantified, so an implementation would need to check where truncation bias becomes negligible for a given $\gamma$.
  • The same deterministic-estimator-plus-quantum-mean-estimation pattern could be tried for actor-critic or Q-learning updates, but each variant would need its own error decomposition and oracle construction, so it remains an open extension.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes a Quantum Natural Policy Gradient (QNPG) algorithm for infinite-horizon discounted Markov decision processes with general policy parameterization. It replaces classical stochastic NPG estimators with deterministic truncated estimators \hat g_ρ(τ_N|θ) and \hat F_ρ(τ_N|θ), evaluates them coherently on superpositions of length-N trajectories, and feeds them to a quantum variance-reduction subroutine (QVarianceReduce). The main result, Theorem 3, claims that under Assumptions 1–3 and with H = O(log(1/ε)), N = O(log(1/ε)), K = O(1/ε), σ_g^2 = O(ε), and σ_F^2 = O((1−γ)^4 ε), Algorithm 1 guarantees J*_ρ − (1/K)Σ_k E[J_ρ(θ_k)] ≤ √ε_bias + ε with \tilde O(ε^{−1.5}) sample complexity and O(ε^{−1}) iteration complexity, improving on the classical \tilde Ω(ε^{−2}) lower bound.

Significance. If the theorem holds as stated, this is a significant result: it would be the first quantum speedup for model-free infinite-horizon policy gradient optimization with general policy parameterization. The paper's main analytical contribution is a bias–variance decomposition for deterministic truncated trajectory estimators, and the proof structure is largely explicit in the appendices. The paper correctly builds on external quantum mean estimation results (Cornelissen et al. 2022; Sidford and Zhang 2024) and shows a leading-order rate that is internally consistent. However, the implementation gap concerning the reward oracle and a register-counting defect in the trajectory oracle construction must be resolved before the central claim can be accepted at face value.

major comments (4)
  1. [Section 2.3 and Appendix A.2, Eqs. (4)–(6), (14), (17)] The oracle model defines only the transition oracle U_P, the initial-state oracle U_ρ, and the policy oracle Π; no quantum reward oracle is defined or assumed. Yet the estimator \hat g_ρ(τ_N|θ) in Eq. (14) requires r(s_n,a_n), and the construction of U_g in Eq. (17) and Appendix A.2 explicitly relies on r(·,·) being classically computable and evaluates it in superposition. If the reward is not available to the learner, Algorithm 1 cannot form \tilde g_h, and Theorem 3's \tilde O(ε^{−1.5}) guarantee is not implementable as stated. This also creates an asymmetry with the classical \tilde Ω(ε^{−2}) MDP-query lower bound, which does not grant analytic knowledge of the reward. The authors need to add a quantum reward oracle to the model and count its queries, or state and defend a different modeling assumption; the rate claim cannot be evaluated until this is resolved.
  2. [Appendix A.1, Eq. (7)] The construction of U_P(τ_N) allocates N state registers S_0, …, S_{N−1}, but at step t = N−1 the transition oracle U_P writes a superposition over s_N into register S_N, which has not been allocated. The final amplitude includes P(s_N|s_{N−1},a_{N−1}) without a corresponding register, so the unitary is ill-defined as written. Adding an (N+1)-th state register fixes the construction and preserves the O(N) query count, but as written Lemma 4 is not established for the stated registers.
  3. [Appendix D, Eq. (58)] The proof of Lemma 5 derives ||E[\tilde F_h] − F_ρ(θ)||^2 ≤ G^4 γ^{2N}/(1−γ)^2, whereas the lemma statement (33) gives G^4 γ^{2N}. These cannot both be correct: since δ_F = G^2 γ^N from Theorem 1, the squared bias bound should be G^4 γ^{2N}. The proof needs correction, and the constants flowing from this bound into Lemma 6 should be rechecked.
  4. [Lemma 6 and Theorem 3] Lemma 6 states the condition α ≤ μ_F/(56G^4), while the proof requires α ≤ μ_F/[4(6σ_F^2 + 6δ_F^2 + 2G^4)]. For the parameter choices in Theorem 3, σ_F^2 and δ_F^2 tend to 0 as ε tends to 0, so the stated condition is sufficient for the asymptotic rate, but the mismatch between the lemma statement and the proof should be cleaned up and the exact dependency on σ_F, σ_g should be made explicit.
minor comments (4)
  1. [Eq. (24)] The definition of ω*_θ writes arg max over L^ν_{π_θ^ρ}(ω, θ), but Eq. (12) defines L as a minimization objective; this should be arg min.
  2. [Algorithm 1, line 7] The notation ∇_ω \tilde L(ω, τ_N) retains an explicit dependence on τ_N, but the quantities \tilde F_h and \tilde g_h are already Monte-Carlo estimates over trajectories; using τ_N here is confusing.
  3. [Theorem 1, Eq. (30)] The variance bound Var(\hat g_ρ) ≤ dG^2/(1−γ)^4 is conservative: the pointwise norm bound ||\hat g_ρ|| ≤ G/(1−γ)^2 already gives a d-independent bound. Since the d factor enters the final complexity through Lemma 1, the authors should state which variance convention is used and confirm the dimension dependence is intentional.
  4. [Section 1.1] The statement that this is the first quantum model-free RL algorithm with theoretical guarantees is too strong given existing quantum policy gradient and tabular QRL results; it should be qualified to general-parameterization infinite-horizon policy gradient methods.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the quantum speedup is imported from external mean-estimation results, and the classical NPG outer-loop bound is an independent prior theorem.

full rationale

The derivation chain is not circular. The outer-loop convergence decomposition is imported as Lemma 3 from Mondal and Aggarwal 2024a; although this is a self-citation (Aggarwal is a coauthor), it is a proved theorem of a separate prior paper that does not assume the quantum result, so it is real evidence rather than a circular premise. The truncation bias and variance analysis in Theorem 1 is derived directly from the definitions in (14) and (15), with no fitted parameter being renamed as a prediction. The claimed speedup rests on QuantumMeanEstimation (Cornelissen et al. 2022, cited as Lemma 1) and QVarianceReduce (Sidford and Zhang 2024, cited as Lemma 7), both external to the authors; the inner-loop query counts follow by substituting the variance bounds (30) into these external lemmas. The final parameter choices H, N, K, sigma_g^2, and sigma_F^2 are standard error-balancing choices, not predictions of fitted quantities. The only substantive gap we identified is not circularity: Section 2.3 defines quantum access only via U_P, U_rho, and Pi, yet constructing U_g in (17) requires rewards, and Appendix A.2 silently states 'Because pi_theta(·|·), r(·, ·), and discounting can be computed classically in polynomial time, these functions are straightforwardly computable given full knowledge of tau_N.' This is a missing modeling assumption about the learner knowing the reward function, which affects implementability and the fairness of the comparison with the classical MDP-query lower bound, but it does not make Theorem 3 equivalent to its inputs by construction. Minor technical defects (e.g., the missing S_N register in Appendix A.1 and the typo in Lemma 5's bound) are correctness issues, not circularity.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

No numerical data are fitted: H, N, K, learning rates, and target variances are analytic choices made as functions of epsilon and problem constants in Theorem 3, not free parameters fitted to data. No new physical entities are postulated: U_F and U_g are unitary implementations of classically computable functions over existing trajectory registers, not new forces, particles, dimensions, or conserved quantities. The load-bearing axioms are the three explicit assumptions plus the unstated reward-computability premise and the imported quantum mean estimation results.

assumptions (6)
  • domain assumption The score function nabla_theta log pi_theta(a|s) is G-Lipschitz and B-smooth (Assumption 1).
    Invoked in Lemma 2 and all bias/variance bounds; standard in the cited policy gradient literature and holds for Gaussian policies.
  • domain assumption The compatible function approximation error at the optimal policy is bounded by epsilon_bias (Assumption 2).
    Used to turn inner-loop NPG convergence into a global optimality gap; zero for softmax, small for expressive neural policies per cited work.
  • domain assumption The Fisher information matrix satisfies F_rho(theta) >= mu_F I (Assumption 3).
    Needed for strong convexity of the inner NPG problem in Lemma 6; the paper calls this the Fisher Non-degeneracy set.
  • ad hoc to paper Reward r(s,a), score gradients, and discounting are efficiently classically computable in superposition without an additional reward oracle.
    Not stated as an assumption. Section 2.3 defines only U_P and U_rho, but Appendix A.2 says r and pi are classically computable to build U_g and U_F.
  • standard math Near-optimal quantum multivariate mean estimation and quantum variance reduction (Cornelissen et al. 2022; Sidford and Zhang 2024) work as stated.
    The main speedup is imported from these external results; Lemma 1 and Lemma 7 are used without proof.
  • domain assumption The policy is quantum-evaluatable as a unitary Pi; any classical policy can be converted to such a unitary.
    Needed to create coherent trajectory superpositions in Eq. (7).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Accelerating Quantum Reinforcement Learning with a Quantum Natural Policy Gradient Based Approach." pith.science (2026). https://pith.science/paper/WQ4FZLMH

@misc{pith2026250116243,
  author       = {Pith},
  title        = {Pith review of: Accelerating Quantum Reinforcement Learning with a Quantum Natural Policy Gradient Based Approach},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WQ4FZLMH}},
  note         = {Machine review of arXiv:2501.16243}
}
abstract

We address the problem of quantum reinforcement learning (QRL) under model-free settings with quantum oracle access to the Markov Decision Process (MDP). This paper introduces a Quantum Natural Policy Gradient (QNPG) algorithm, which replaces the random sampling used in classical Natural Policy Gradient (NPG) estimators with a deterministic gradient estimation approach, enabling seamless integration into quantum systems. While this modification introduces a bounded bias in the estimator, the bias decays exponentially with increasing truncation levels. This paper demonstrates that the proposed QNPG algorithm achieves a sample complexity of $\tilde{\mathcal{O}}(\epsilon^{-1.5})$ for queries to the quantum oracle, significantly improving the classical lower bound of $\tilde{\mathcal{O}}(\epsilon^{-2})$ for queries to the MDP.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 34 canonical work pages

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    M., Lee, J

    Agarwal, A., Kakade, S. M., Lee, J. D., and Mahajan, G. Optimality and approximation with policy gradient methods in markov decision processes. In Abernethy, J. and Agarwal, S. (eds.), Proceedings of Thirty Third Conference on Learning Theory, volume 125 of Proceedings of Machine Learning Research, pp.\ 64--66. PMLR, 09--12 Jul 2020

    work page 2020

  3. [3]

    M., Lee, J

    Agarwal, A., Kakade, S. M., Lee, J. D., and Mahajan, G. On the theory of policy gradient methods: Optimality, approximation, and distribution shift. Journal of Machine Learning Research, 22 0 (98): 0 1--76, 2021

    2021

  4. [4]

    O., Ghosh, A., and Aggarwal, V

    Al-Abbasi, A. O., Ghosh, A., and Aggarwal, V. Deeppool: Distributed model-free algorithm for ride-sharing using deep reinforcement learning. IEEE Transactions on Intelligent Transportation Systems, 20 0 (12): 0 4714--4727, 2019

    work page 2019

  5. [5]

    S., and Aggarwal, V

    Bai, Q., Bedi, A. S., and Aggarwal, V. Achieving zero constraint violation for constrained reinforcement learning via conservative natural policy gradient primal-dual algorithm. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp.\ 6737--6744, 2023

    work page 2023

  6. [6]

    and Bartlett, P

    Baxter, J. and Bartlett, P. L. Infinite-horizon policy-gradient estimation. journal of artificial intelligence research, 15: 0 319--350, 2001

    work page 2001

  7. [7]

    Quantum amplitude amplification and estimation

    Brassard, G., Hoyer, P., Mosca, M., and Tapp, A. Quantum amplitude amplification and estimation. Contemporary Mathematics, 305: 0 53--74, 2002

    work page 2002

  8. [8]

    Quantum bandits

    Casal \'e , B., Di Molfetta, G., Kadri, H., and Ralaivola, L. Quantum bandits. Quantum Machine Intelligence, 2: 0 1--7, 2020

    work page 2020

Show all 40 references
  1. [9]

    Near-optimal quantum algorithms for multivariate mean estimation

    Cornelissen, A., Hamoudi, Y., and Jerbi, S. Near-optimal quantum algorithms for multivariate mean estimation. In Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing, pp.\ 33--43, 2022

    2022

  2. [10]

    Quantum reinforcement learning

    Dong, D., Chen, C., Li, H., and Tarn, T.-J. Quantum reinforcement learning. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 38 0 (5): 0 1207--1220, 2008

    2008

  3. [11]

    M., and Briegel, H

    Dunjko, V., Taylor, J. M., and Briegel, H. J. Advances in quantum reinforcement learning. In 2017 IEEE International Conference on Systems, Man, and Cybernetics (SMC), pp.\ 282--287. IEEE, 2017

    2017

  4. [12]

    U., and Aggarwal, V

    Ganesh, S., Chen, J., Mondal, W. U., and Aggarwal, V. Order-optimal global convergence for actor-critic with general policy and neural critic parametrization. In The 41st Conference on Uncertainty in Artificial Intelligence, 2025

    2025

  5. [13]

    Quantum speedups in regret analysis of infinite horizon average-reward markov decision processes

    Ganguly, B., Xu, Y., and Aggarwal, V. Quantum speedups in regret analysis of infinite horizon average-reward markov decision processes. arXiv preprint arXiv:2310.11684, 2023

    2023 arXiv

  6. [14]

    S., Wang, D., and Aggarwal, V

    Gaur, M., Bedi, A. S., Wang, D., and Aggarwal, V. Closing the gap: Achieving global convergence (last iterate) of actor-critic under markovian sampling with neural network parametrization. arXiv preprint arXiv:2405.01843, 2024

    2024 arXiv

  7. [15]

    W., McKee, J., Hager, G., Aggarwal, V., Xue, Y., et al

    Gonzalez, G., Balakuntala, M., Agarwal, M., Low, T., Knoth, B., Kirkpatrick, A. W., McKee, J., Hager, G., Aggarwal, V., Xue, Y., et al. Asap: A semi-autonomous precise system for telesurgery during communication delays. IEEE Transactions on Medical Robotics and Bionics, 5 0 (1...

    2023

  8. [16]

    and Rudolph, T

    Grover, L. and Rudolph, T. Creating superpositions that correspond to efficiently integrable probability distributions. quant-ph/0208112, 2002

    2002 arXiv

  9. [17]

    Grover, L. K. A fast quantum mechanical algorithm for database search. In Proceedings of the twenty-eighth annual ACM symposium on Theory of computing, pp.\ 212--219, 1996

    1996

  10. [18]

    Quantum sub-gaussian mean estimator

    Hamoudi, Y. Quantum sub-gaussian mean estimator. In 29th Annual European Symposium on Algorithms (ESA 2021). Schloss Dagstuhl-Leibniz-Zentrum f \"u r Informatik, 2021

    2021

  11. [19]

    M., Nautrup, H

    Jerbi, S., Trenkwalder, L. M., Nautrup, H. P., Briegel, H. J., and Dunjko, V. Quantum enhancements for deep reinforcement learning in large spaces. PRX Quantum, 2 0 (1): 0 010328, 2021

    2021

  12. [20]

    Quantum policy gradient algorithms

    Jerbi, S., Cornelissen, A., Ozols, M., and Dunjko, V. Quantum policy gradient algorithms. In 18th Conference on the Theory of Quantum Computation, Communication and Cryptography, 2023

    2023

  13. [21]

    An improved analysis of (variance-reduced) policy gradient and natural policy gradient methods

    Liu, Y., Zhang, K., Basar, T., and Yin, W. An improved analysis of (variance-reduced) policy gradient and natural policy gradient methods. Advances in Neural Information Processing Systems, 33: 0 7624--7636, 2020

    2020

  14. [22]

    Mondal, W. U. and Aggarwal, V. Improved sample complexity analysis of natural policy gradient algorithm with general parameterization for infinite horizon discounted reward markov decision processes. In International Conference on Artificial Intelligence and Statistics, pp.\ 3...

    2024

  15. [23]

    Mondal, W. U. and Aggarwal, V. Sample-efficient constrained reinforcement learning with general parameterization. Advances in Neural Information Processing Systems, 37: 0 68380--68405, 2024 b

    2024

  16. [24]

    Quantum speedup of monte carlo methods

    Montanaro, A. Quantum speedup of monte carlo methods. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 471 0 (2181): 0 20150301, 2015

    2015

  17. [25]

    Nielsen, M. A. and Chuang, I. L. Quantum computation and quantum information. Cambridge university press, 2010

    2010

  18. [26]

    D., Dunjko, V., Makmal, A., Martin-Delgado, M

    Paparo, G. D., Dunjko, V., Makmal, A., Martin-Delgado, M. A., and Briegel, H. J. Quantum speedup for active learning agents. Physical Review X, 4 0 (3): 0 031002, 2014

    2014

  19. [27]

    and Schaal, S

    Peters, J. and Schaal, S. Natural actor-critic. Neurocomputing, 71 0 (7-9): 0 1180--1190, 2008

    2008

  20. [28]

    and Zhang, C

    Sidford, A. and Zhang, C. Quantum speedups for stochastic optimization. Advances in Neural Information Processing Systems, 36, 2024

    2024

  21. [29]

    S., McAllester, D., Singh, S., and Mansour, Y

    Sutton, R. S., McAllester, D., Singh, S., and Mansour, Y. Policy gradient methods for reinforcement learning with function approximation. Advances in neural information processing systems, 12, 1999

    1999

  22. [30]

    P., Yu, D., and Aggarwal, V

    Tamboli, D., Chen, J., Jotheeswaran, K. P., Yu, D., and Aggarwal, V. Reinforced sequential decision-making for sepsis treatment: The posnegdm framework with mortality classifier and transformer. IEEE Journal of Biomedical and Health Informatics, 2024

    2024

  23. [31]

    Quantum multi-armed bandits and stochastic linear bandits enjoy logarithmic regrets

    Wan, Z., Zhang, Z., Li, T., Zhang, J., and Sun, X. Quantum multi-armed bandits and stochastic linear bandits enjoy logarithmic regrets. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 37, pp.\ 10087--10094, 2023

    2023

  24. [32]

    Quantum algorithms for reinforcement learning with a generative model

    Wang, D., Sundaram, A., Kothari, R., Kapoor, A., and Roetteler, M. Quantum algorithms for reinforcement learning with a generative model. In International Conference on Machine Learning, pp.\ 10916--10926. PMLR, 2021 a

    2021

  25. [33]

    Wang, D., You, X., Li, T., and Childs, A. M. Quantum exploration algorithms for multi-armed bandits. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp.\ 10102--10110, 2021 b

    2021

  26. [34]

    Wang, D., Gao, N., Liu, D., Li, J., and Lewis, F. L. Recent progress in reinforcement learning and adaptive dynamic programming for advanced control applications. IEEE/CAA Journal of Automatica Sinica, 2023

    2023

  27. [35]

    Neural policy gradient methods: Global optimality and rates of convergence

    Wang, L., Cai, Q., Yang, Z., and Wang, Z. Neural policy gradient methods: Global optimality and rates of convergence. In International Conference on Learning Representations, 2019

    2019

  28. [36]

    Wiedemann, S., Hein, D., Udluft, S., and Mendl, C. B. Quantum policy iteration via amplitude estimation and grover search--towards quantum advantage for reinforcement learning. Transactions on Machine Learning Research, 2022

    2022

  29. [37]

    Quantum heavy-tailed bandits

    Wu, Y., Guan, C., Aggarwal, V., and Wang, D. Quantum heavy-tailed bandits. arXiv preprint arXiv:2301.09680, 2023

    2023 arXiv

  30. [38]

    Sample efficient policy gradient methods with recursive variance reduction

    Xu, P., Gao, F., and Gu, Q. Sample efficient policy gradient methods with recursive variance reduction. In International Conference on Learning Representations, 2019

    2019

  31. [39]

    Global convergence of policy gradient methods to (almost) locally optimal policies

    Zhang, K., Koppel, A., Zhu, H., and Basar, T. Global convergence of policy gradient methods to (almost) locally optimal policies. SIAM Journal on Control and Optimization, 58 0 (6): 0 3586--3612, 2020

    2020

  32. [40]

    Provably efficient exploration in quantum reinforcement learning with logarithmic worst-case regret

    Zhong, H., Hu, J., Xue, Y., Li, T., and Wang, L. Provably efficient exploration in quantum reinforcement learning with logarithmic worst-case regret. In Forty-first International Conference on Machine Learning, 2024

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.