A hybrid learning agent for episodic learning tasks with unknown target distance

Oliver Sefrin; Sabine W\"olk

REVIEW 2 major objections 5 minor 56 references

A hybrid learning agent for episodic learning tasks with unknown target distance

T0 review · 2 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a hybrid quantum-classical reinforcement-learning agent with a probabilistic episode-length-doubling rule finds the first reward with 27-42 percent fewer total actions than the probabilistic classical agent when…

desk verdict A neat and carefully hedged extension of the hybrid quantum-classical RL agent, but the printed amplitude-amplification formula in Appendix B is wrong in a load-bearing way, so the quantitative results need verification before they can be trusted. read the letter →

arxiv 2412.13686 v1 pith:NFQ2W5JB submitted 2024-12-18 quant-ph

Oliver Sefrin , Sabine Wölk This is my paper · ORCID

classification quant-ph MSC 68T0581P68

keywords quantumreinforcementlearninghybridquantum-classicalagentamplitudeamplificationepisodictasksunknowntargetdistanceepisodelengthadaptationGridworldnavigationfirst-rewardsearch

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Episodic reinforcement learning normally requires knowing how long an episode may last, and the hybrid quantum-classical agent studied here previously needed a fixed episode length to run amplitude amplification. This paper argues that a probabilistic rule which doubles the current episode length whenever a trigger tied to the search algorithm's uncertainty parameter fires removes that requirement. In simulated Gridworld mazes, where an untrained agent with a uniform policy searches for its first reward, the adapted hybrid strategy uses 27-42 percent fewer total interaction steps than a classical agent with the same doubling rule once the outer walls are at distance 8 or more, and it also beats the unrestricted classical random-walk agent for wall distances of 16 or more. The hybrid tends to terminate at shorter episode lengths, which the paper reads as evidence that it finds shorter paths. The paper notes that the proven quasi-quadratic speedup in episode queries does not carry over to this new total-action metric.

What carries the argument

The load-bearing mechanism is the probabilistic episode-length-doubling rule $\varphi_L(m)=2\log(m)/(L\log|A|)$, where $L$ is the current episode length, $|A|$ the number of actions, and $m$ the interval bound of an expanding-interval amplitude-amplification variant designed for an unknown number of solutions. In each loop the agent doubles $L$ with probability $\varphi_L(m)$, resets $m$ to 1 when doubling occurs, samples an integer $k$ uniformly from $[0,m)$, applies $k$ Grover iterations, and then runs a classical episode of length $L$ to read out the reward. The doubling probability reaches one exactly when the amplitude-amplification search reaches its critical stage, so the schedule needs no additional hyperparameters. The paper's figure of merit is the total number of actions $N_{\mathrm{act}}$, which counts $2kL$ for the quantum queries and $L$ for the classical verification episode, so the comparison reflects actual interaction cost rather than just episode count.

What would settle it

A single reproducible run of the 5x5 Gridworld with outer wall distance 64 using the standard formula $p_{\mathrm{AA}}(L,k)=\sin^2((2k+1)\arcsin(\sqrt{p_{\mathrm{init}}(L)}))$ should recover the reported $N_{\mathrm{act}}=499\pm 3$ for the probabilistic hybrid; if the printed inverse-root formula was used instead, the simulated success probabilities would not describe amplitude amplification and the values in Table A1 would not be reproducible.

Watch

Extended reading notes

Core claim

The central claim is that the hybrid agent for quantum-accessible reinforcement learning can be extended from strictly episodic environments with a fixed episode length to deterministic episodic tasks with an unknown target distance, by interleaving amplitude amplification with a stochastic episode-length-doubling schedule. According to the simulations, the adapted hybrid finds the first reward faster than a classical agent using the same probabilistic schedule in nearly every tested Gridworld layout, with 27-42 percent fewer total actions for outer wall distances of 8 or more, and it overtakes the unrestricted classical agent for wall distances of 16 or more. The most frequent terminal episode lengths of the hybrid are the powers of two closest to the fixed episode length that would be optimal with full knowledge, which the paper presents as evidence that the doubling rule converges to efficient lengths without overshooting. The paper also states explicitly that under the total-action figure of merit the quadratic scaling advantage of the original hybrid agent no longer applies.

Load-bearing premise

The load-bearing premise is that the simulated success probabilities are those of real amplitude amplification; the formula printed in Appendix B uses the inverse square root of the initial success probability, while the standard amplitude-amplification formula uses the square root, so the hybrid performance numbers depend on the implementation having followed the standard formula despite the printed one.

Editorial extensions

If this is right

If the simulation results hold, the hybrid agent can be applied to episodic learning tasks where the target distance is unknown, without adding hyperparameters beyond the original algorithm.
For Gridworld layouts with outer wall distance at least 8, the probabilistic hybrid uses 27-42 percent fewer total interaction steps than the probabilistic classical strategy, and the gap grows with the wall distance.
For wall distances of 16 or larger, the hybrid also beats the unrestricted classical agent that searches without episode resets, and for the largest tested configuration (9x9 cells with wall distance 64) it does so by more than an order of magnitude.
The narrower terminal episode-length distribution implies the hybrid tends to find shorter action sequences, which can speed up subsequent learning after the first reward is found.
The price of dropping the fixed episode length is that the proven quasi-quadratic speedup in episode queries no longer applies to the total-action figure of merit.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: the same doubling schedule should extend directly to mazes with absorbing but unrewarded terminal walls, since the paper argues the strategy is tailored to slowly rising success probabilities; simulating that variant would test the claim.
Editorial inference: in grids of dimension three or higher, the unrestricted classical random walk need not reach every target with probability one, so the hybrid's advantage over the unrestricted strategy would likely become even more pronounced than in the two-dimensional cases shown.
Editorial inference: one testable consequence of the paper's account is that, under the original episode-query metric, the hybrid should still show a large saving over the probabilistic classical agent for small episode lengths, even though that saving is partially consumed by the classical verification episodes in the total-action metric.
Editorial inference: the numerical results could be made fully reproducible by releasing the Monte-Carlo code that estimates $p_{\mathrm{init}}(L)$ and the amplitude-amplification sampling routine; re-implementing it with the standard formula would settle which version of the Appendix B equation the reported numbers actually follow.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The stochastic episode-length-doubling idea is the real contribution here. Coupling the doubling probability to Boyer's m parameter so that the episode length grows in step with the critical stage of amplitude amplification is a natural adaptation, and doing it without introducing extra hyperparameters is a plus. The new figure of merit, total interaction steps Nact, is the right metric for variable-length episodes, and the two classical baselines are sensible. The simulation work is substantial: 10,000 runs per configuration, standard errors reported, and conclusions that stay close to what the data actually show. I believe the qualitative story—hybrid helps when the state space is large and initial success probabilities are low, while the unrestricted classical agent wins when walls keep the random walk close to the target—will survive scrutiny.

The soft spots are real, though. The amplitude amplification formula in Appendix B is printed as sin^2((2k+1) arcsin(p_init(L)^{-1/2})). The standard Brassard result is sin^2((2k+1) arcsin(sqrt(p_init(L)))). For p_init < 1, the printed version takes arcsin of a number greater than 1, which is not a real success probability. Since every hybrid-agent number in Tables A1 and Figures 4-7 comes from a simulation using this step, the central quantitative comparison is unverifiable as written. I suspect it is a typo and the code used the standard formula, but without code or data release a reader cannot tell. That needs to be fixed or the code released. This is not a stylistic complaint; it is the load-bearing simulation equation.

Second, the relationship to Hamann et al. 2021, whose title promises "beyond strictly epochal environments," is never discussed. The paper cites it only as an extension involving changing oracles. If that work already handles non-strictly episodic settings, then the claimed open question is at least partially answered, and the novelty claim needs to be sharpened. If not, the authors should say why not. As it stands, the novelty boundary is unclear.

Third, no code or data are provided for the 10,000-run simulations. That would be a minor point on its own, but combined with the formula issue it becomes more significant.

Who is this for? Researchers working on quantum-accessible reinforcement learning, especially the hybrid-agent line. The idea is worth engaging with, and the paper deserves a serious referee. My recommendation: send it to peer review, but require the authors to correct the formula, clarify the relation to Hamann et al. 2021, and ideally release the simulation code so the tables can be reproduced.

Referee Report

2 major / 5 minor

Summary. The paper extends the hybrid quantum-classical reinforcement-learning agent of Hamann and Wölk (2022) to episodic tasks in which the target distance, and hence the necessary episode length, is unknown in advance. The proposed mechanism couples the episode length L to the Boyer-iteration parameter m: L is doubled with probability log(m)/log(sqrt(|A|^L)), and m is reset upon doubling. The authors compare the resulting 'probabilistic hybrid' agent against a classical agent using the same episode-length-doubling rule and against an unrestricted classical random walk, in Gridworld layouts with varying base size and outer-wall distance. They count the total number of interaction steps Nact until the first reward is found, using 10,000 Monte Carlo runs per configuration. The main reported results are that, for outer-wall distances dwall >= 8, the probabilistic hybrid strategy uses 27% to 42% fewer interaction steps than the probabilistic classical strategy, and that for dwall >= 16 it also beats the unrestricted classical strategy, while the unrestricted classical agent remains preferable for small dwall.

Significance. If the numerical results are correct, the paper makes a useful and well-motivated extension: it removes the requirement to know the episode length in advance, at the price of losing the proven quadratic query-complexity speedup and replacing it with an empirical advantage measured in total interaction steps. The simulation study is extensive (10,000 runs per configuration, standard errors reported, multiple Gridworld shapes), and the conclusions are generally carefully hedged. The paper does not contain formal proofs or machine-checked code; its credibility rests entirely on the correctness of the simulation pipeline described in Appendix B, and that pipeline currently contains a load-bearing error in the printed amplitude-amplification formula. Because the central quantitative claims are generated from that formula and no code or data are provided, the results cannot be verified as written.

major comments (2)

[Appendix B, amplitude amplification formula] The printed formula is pAA(L,k) = sin^2((2k+1) arcsin[p_init(L)^(-1/2)]). The standard result of Brassard et al. (2002) is sin^2((2k+1) arcsin[p_init(L)^(1/2)]). For p_init(L) < 1, the printed expression takes arcsin of a number larger than 1, which is undefined over the reals and cannot describe a success probability. Every hybrid-agent number in Table A1, Figures 4-7, and Appendix C is generated from this formula through the Monte Carlo sampling described in Appendix B. If the implementation followed the printed formula, the reported hybrid advantage is not the result of amplitude amplification; if the implementation used the standard formula, the manuscript contains a load-bearing misprint. Since no code or data are released, a reader cannot determine which version produced the results. I request that the authors correct the formula and either release the simulation code/data or otherwise demonstrate that the corrected formula reproduces the reported values in Table A1 and Figures 4-7.
[Section 5, conclusion claim (ii)] The conclusion states that the proposed hybrid agent 'can find shorter solutions' than the considered classical agents. The only evidence presented is the distribution of terminal episode lengths (Figures 5-7), which is an upper bound on the actual rewarded-path length, not the length itself. The actual distribution of found solution lengths is not reported. The conclusion should either report the measured solution-length distributions or be softened to a statement about lower terminal episode lengths, which merely suggest, rather than establish, shorter solutions.

minor comments (5)

[Section 4.2, paragraph on dwall = 8 and larger] The statement that the hybrid strategy uses '27% to 42% fewer actions' should explicitly identify the baseline as the probabilistic classical strategy; otherwise a reader may infer the comparison is with the unrestricted classical strategy.
[Section 3.1, Algorithm 1 and surrounding text] The paper says the method introduces 'no additional hyperparameters', but lambda = 5/4 is a free parameter (along with the initial L = 1). Please clarify whether lambda is inherited from Boyer's algorithm and add a sensitivity study or at least a remark on the role of this parameter.
[Section 3.1, paragraph after Algorithm 1] The lower bound pmin = |A|^(-L) is valid only when at least one rewarded action sequence exists at the current episode length. For L below the shortest path length this is not the case. The algorithm still makes progress by doubling L, but the text should state explicitly that the Boyer critical-stage guarantee applies only after L reaches the minimum successful length.
[Appendix A, Table A1 caption] There is a typographical artifact in the caption ('T able A1') that should be corrected.
[Appendix B, simulation details] The standard errors reported in Table A1 reflect the randomness of the runs, but the uncertainty in the Monte Carlo estimate of p_init(L) is not propagated into the reported Nact standard errors; this should be acknowledged or quantified.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the performance comparison is a simulation from externally established amplitude amplification, with no fitted parameter renamed as a prediction; the Appendix B formula issue is a correctness concern, not circularity.

full rationale

The paper's central claim—that the probabilistic hybrid agent finds the first reward with fewer total interaction steps than the probabilistic or unrestricted classical strategies in certain Gridworld layouts—is an empirical simulation result. The inputs are (i) the hybrid-agent framework of Hamann and Wölk (2022), which has independent experimental support in Saggio et al. (2021) and is cited rather than re-derived; (ii) the standard amplitude-amplification success formula of Brassard et al. (2002); (iii) Monte Carlo estimates of the initial success probability pinit(L) for each layout; and (iv) the explicitly stated cost model where each Grover iteration adds 2kL environment queries and each classical episode adds L. None of these inputs is fitted to the reported Nact values, and the episode-length-doubling rule (Algorithm 1, Eq. 6) is a design choice stated in the paper, not a hidden parameter fitted to the results. The observation that the hybrid agent terminates near the optimal fixed episode length is flagged by the authors themselves as an open question (Section 5), not presented as a forced theorem. The only concrete defect found in review—the Appendix B formula as printed, pAA(L,k)=sin2((2k+1) arcsin[pinit(L)^(-1/2)]), which inverts the standard arcsin(sqrt(pinit(L)))—is a correctness/verifiability issue (and a likely misprint), not a circularity: the printed equation is not equivalent to its inputs by construction, and there is no fitted parameter renamed as a prediction. Therefore the circularity score is 0.

Assumptions & free parameters 2 free parameters · 5 assumptions · 0 invented entities

The central simulation rests on several domain assumptions about the RL environment (determinism, uniform initial policy, existence of a rewarded sequence at each probed length) and on a standard but apparently mistyped amplitude amplification formula. No code or data were shipped to check the implementation, so the ledger cannot be closed by artifacts.

free parameters (2)

lambda (interval growth factor) = 5/4
Growth factor for the interval bound m in Boyer's algorithm (Algorithm 1, line 3). It controls how quickly m approaches the critical stage and therefore how often the episode length doubles; no sensitivity analysis is reported despite the results depending on it.
initial episode length L_start = 1
Algorithm 1, line 1 starts adaptation from episode length 1. Episode lengths then grow as powers of two, and the choice influences how many early unsolvable episodes are played. It is a design choice rather than a fitted value.

assumptions (5)

domain assumption The environment is deterministic with state transition function S and reward function R.
Section 2.1 (eqs. 2-3) assumes deterministic responses, inherited from the DSE environment class in Hamann and Wölk 2022. This justifies the unitary environment model used for amplitude amplification.
domain assumption The initial policy is uniform, pi(a) = 1/|A| for all actions a.
Section 3.1 assumes a uniform initial policy to set pmin = |A|^{-L} and to compute pinit(L) as the fraction of length-L action sequences reaching the target. The doubling probability phi_L(m) and all simulation results depend on this.
ad hoc to paper At least one rewarded action sequence exists at the current episode length for the pmin bound to be valid.
Section 3.1 states pmin = |A|^{-L} 'assuming ... at least one rewarded action sequence at the current episode length'. For episode lengths below the minimal path length this fails, and the algorithm relies on doubling until a rewarded sequence becomes reachable. This assumption underlies the critical-stage calculation.
standard math Amplitude amplification success probability follows the Brassard et al. 2002 formula.
Appendix B computes the amplified success probability with the AA equation and uses it to sample simulation outcomes. This is standard quantum search theory, but the formula as printed contains an apparent inverted exponent.
domain assumption Each amplitude amplification iteration costs two environment queries (alpha_k = 2).
Footnote on page 4 and Section 4.1 count 2*L environment unitary queries per AA iteration because intermediate percepts are action-dependent in Gridworld. The Nact comparison with classical agents relies on treating each query as equivalent to one interaction step.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A hybrid learning agent for episodic learning tasks with unknown target distance." pith.science (2026). https://pith.science/paper/NFQ2W5JB

@misc{pith2026241213686,
  author       = {Pith},
  title        = {Pith review of: A hybrid learning agent for episodic learning tasks with unknown target distance},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NFQ2W5JB}},
  note         = {Machine review of arXiv:2412.13686}
}

read the original abstract

The "hybrid agent for quantum-accessible reinforcement learning", as defined in (Hamann and W\"olk, 2022), provides a proven quasi-quadratic speedup and is experimentally tested. However, the standard version can only be applied to episodic learning tasks with fixed episode length. In many real-world applications, the information about the necessary number of steps within an episode to reach a defined target is not available in advance and especially before reaching the target for the first time. Furthermore, in such scenarios, classical agents have the advantage of observing at which step they reach the target. Whether the hybrid agent can provide an advantage in such learning scenarios was unknown so far. In this work, we introduce a hybrid agent with a stochastic episode length selection strategy to alleviate the need for knowledge about the necessary episode length. Through simulations, we test the adapted hybrid agent's performance versus classical counterparts. We find that the hybrid agent learns faster than corresponding classical learning agents in certain scenarios with unknown target distance and without fixed episode length.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

56 extracted references · 8 canonical work pages

[1]

, Bergholm , V

botherref Biamonte , J. , Bergholm , V. : Tensor Networks in a Nutshell (2017). https://arxiv.org/abs/1708.00006 botherref

arXiv 2017
[2]

, Wittek , P

barticle Biamonte , J. , Wittek , P. , Pancotti , N. , Rebentrost , P. , Wiebe , N. , Lloyd , S. : Quantum machine learning . Nature 549 ( 7671 ), 195 -- 202 ( 2017 ) 10.1038/nature23474 barticle

doi:10.1038/nature23474 2017
[3]

, Brassard , G

barticle Boyer , M. , Brassard , G. , H yer , P. , Tapp , A. : Tight bounds on quantum searching . Fortschr. Phys. 46 ( 4-5 ), 493 -- 505 ( 1998 ) 10.1002/(SICI)1521-3978(199806)46:4/5<493::AID-PROP493>3.0.CO;2-P barticle

doi:10.1002/(sici)1521-3978(199806)46:4/5 1998
[4]

, H yer , P

barticle Brassard , G. , H yer , P. , Mosca , M. , Tapp , A. : Quantum amplitude amplification and estimation . Contemp. Math. 305 , 53 -- 74 ( 2002 ) 10.1090/conm/305/05215 barticle

doi:10.1090/conm/305/05215 2002
[5]

, Chubb , C.T

barticle Bridgeman , J.C. , Chubb , C.T. : Hand-waving and interpretive dance: an introductory course on tensor networks . J. Phys. A 50 ( 22 ), 223001 ( 2017 ) 10.1088/1751-8121/aa6dc3 barticle

doi:10.1088/1751-8121/aa6dc3 2017
[6]

, Cuevas , G

barticle Briegel , H.J. , Cuevas , G. : Projective simulation for artificial intelligence . Sci. Rep. 2 ( 1 ), 400 ( 2012 ) 10.1038/srep00400 barticle

doi:10.1038/srep00400 2012
[7]

, Cheung , V

botherref Brockman , G. , Cheung , V. , Pettersson , L. , Schneider , J. , Schulman , J. , Tang , J. , Zaremba , W. : OpenAI Gym (2016). https://arxiv.org/abs/1606.01540 botherref

arXiv 2016
[8]

, Day , A.G.R

barticle Bukov , M. , Day , A.G.R. , Sels , D. , Weinberg , P. , Polkovnikov , A. , Mehta , P. : Reinforcement learning in different phases of quantum control . Phys. Rev. X 8 , 031086 ( 2018 ) 10.1103/PhysRevX.8.031086 barticle

doi:10.1103/physrevx.8.031086 2018

Show all 56 references

[9]

, Arrasmith , A

barticle Cerezo , M. , Arrasmith , A. , Babbush , R. , Benjamin , S.C. , Endo , S. , Fujii , K. , McClean , J.R. , Mitarai , K. , Yuan , X. , Cincio , L. , Coles , P.J. : Variational quantum algorithms . Nat. Rev. Phys. 3 ( 9 ), 625 -- 644 ( 2021 ) 10.1038/s42254-021-00348-9 barticle

2021 doi
[10]

, Larocca , M

botherref Cerezo , M. , Larocca , M. , García-Martín , D. , Diaz , N.L. , Braccia , P. , Fontana , E. , Rudolph , M.S. , Bermejo , P. , Ijaz , A. , Thanasilp , S. , Anschuetz , E.R. , Holmes , Z. : Does provable absence of barren plateaus imply classical simulability? Or, why ...

2024 arXiv
[11]

, Yang , C.-H.H

barticle Chen , S.Y.-C. , Yang , C.-H.H. , Qi , J. , Chen , P.-Y. , Ma , X. , Goan , H.-S. : Variational quantum circuits for deep reinforcement learning . IEEE Access 8 , 141007 -- 141024 ( 2020 ) 10.1109/ACCESS.2020.3010470 barticle

2020
[12]

, Chang , Y.-J

barticle Chen , H.-Y. , Chang , Y.-J. , Liao , S.-W. , Chang , C.-R. : Deep q-learning with hybrid quantum neural network on solving maze problems . Quantum Mach. Intell. 6 ( 1 ), 2 ( 2024 ) 10.1007/s42484-023-00137-w barticle

2024 doi
[13]

, Kerenidis , I

barticle Cherrat , E.A. , Kerenidis , I. , Prakash , A. : Quantum reinforcement learning via policy iteration . Quantum Mach. Intell. 5 ( 2 ), 30 ( 2023 ) 10.1007/s42484-023-00116-1 barticle

2023 doi
[14]

, Buffoni , L

barticle Dalla Pozza , N. , Buffoni , L. , Martina , S. , Caruso , F. : Quantum reinforcement learning: the maze problem . Quant. Mach. Intell. 4 ( 1 ), 11 ( 2022 ) 10.1007/s42484-022-00068-y barticle

2022 doi
[15]

, Chen , C

barticle Dong , D. , Chen , C. , Li , H. , Tarn , T.-J. : Quantum reinforcement learning . IEEE Trans. Syst. Man. Cybern. B Cybern. 38 ( 5 ), 1207 -- 1220 ( 2008 ) 10.1109/TSMCB.2008.925743 barticle

2008
[16]

, Taylor , J.M

barticle Dunjko , V. , Taylor , J.M. , Briegel , H.J. : Quantum-enhanced machine learning . Phys. Rev. Lett. 117 , 130501 ( 2016 ) 10.1103/PhysRevLett.117.130501 barticle

2016 doi
[17]

, Tighineanu , P

barticle F\"osel , T. , Tighineanu , P. , Weiss , T. , Marquardt , F. : Reinforcement learning with neural networks for quantum feedback . Phys. Rev. X 8 , 031084 ( 2018 ) 10.1103/PhysRevX.8.031084 barticle

2018 doi
[18]

, Niu , M.Y

botherref F \"o sel , T. , Niu , M.Y. , Marquardt , F. , Li , L. : Quantum circuit optimization with deep reinforcement learning (2021). https://arxiv.org/abs/2103.07585 botherref

2021 arXiv
[19]

, Wu , Y

botherref Ganguly , B. , Wu , Y. , Wang , D. , Aggarwal , V. , Ganguly , B. , Wu , Y. , Wang , D. , Aggarwal , V. : Quantum Computing Provides Exponential Regret Improvement in Episodic Reinforcement Learning (2023). https://arxiv.org/abs/2302.08617 botherref

2023 arXiv
[20]

: Quantum mechanics helps in searching for a needle in a haystack

barticle Grover , L.K. : Quantum mechanics helps in searching for a needle in a haystack . Phys. Rev. Lett. 79 , 325 -- 328 ( 1997 ) 10.1103/PhysRevLett.79.325 barticle

1997 doi
[21]

, Susto , G.A

barticle Guatto , M. , Susto , G.A. , Ticozzi , F. : Improving robustness of quantum feedback control with reinforcement learning . Phys. Rev. A 110 , 012605 ( 2024 ) 10.1103/PhysRevA.110.012605 barticle

2024 doi
[22]

, Dunjko , V

barticle Hamann , A. , Dunjko , V. , W \"o lk , S. : Quantum-accessible reinforcement learning beyond strictly epochal environments . Quantum Mach. Intell. 3 ( 2 ), 22 ( 2021 ) 10.1007/s42484-021-00049-7 barticle

2021 doi
[23]

, W \"o lk , S

barticle Hamann , A. , W \"o lk , S. : Performance analysis of a hybrid agent for quantum-accessible reinforcement learning . New J. Phys. 24 ( 3 ), 033044 ( 2022 ) 10.1088/1367-2630/ac5b56 barticle

2022 doi
[24]

, Heimann , D

barticle Hohenfeld , H. , Heimann , D. , Wiebe , F. , Kirchner , F. : Quantum deep reinforcement learning for robot navigation tasks . IEEE Access 12 , 87217 -- 87236 ( 2024 ) 10.1109/ACCESS.2024.3417808 barticle

2024
[25]

, Patil , P

barticle Huggins , W. , Patil , P. , Mitchell , B. , Whaley , K.B. , Stoudenmire , E.M. : Towards quantum machine learning with tensor networks . Quantum Sci. Technol. 4 ( 2 ), 024001 ( 2019 ) 10.1088/2058-9565/aaea94 barticle

2019 doi
[26]

, Gyurik , C

bchapter Jerbi , S. , Gyurik , C. , Marshall , S.C. , Briegel , H.J. , Dunjko , V. : Parametrized quantum policies for reinforcement learning . In: Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 20...

2021
[27]

: Quantum measurements and the Abelian Stabilizer Problem (1995)

botherref Kitaev , A.Y. : Quantum measurements and the Abelian Stabilizer Problem (1995). https://arxiv.org/abs/quant-ph/9511026 botherref

1995 arXiv
[28]

, Towers , M

botherref Kwiatkowski , A. , Towers , M. , Terry , J. , Balis , J.U. , De Cola , G. , Deleu , T. , Goul\ a o , M. , Kallinteris , A. , Krimmel , M. , KG , A. , Perez-Vicente , R. , Pierr\' e , A. , Schulhoff , S. , Tai , J.J. , Tan , H. , Younis , O.G. : Gymnasium: A Standard ...

2024 arXiv
[29]

, Hunt , J.J

botherref Lillicrap , T.P. , Hunt , J.J. , Pritzel , A. , Heess , N. , Erez , T. , Tassa , Y. , Silver , D. , Wierstra , D. : Continuous control with deep reinforcement learning (2019). https://arxiv.org/abs/1509.02971 botherref

2019 arXiv
[30]

: Optimizing Quantum Variational Circuits with Deep Reinforcement Learning (2022)

botherref Lockwood , O. : Optimizing Quantum Variational Circuits with Deep Reinforcement Learning (2022). https://arxiv.org/abs/2109.03188 botherref

2022 arXiv
[31]

, Si , M

bchapter Lockwood , O. , Si , M. : Reinforcement learning with quantum variational circuits . In: Proceedings of the AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment , Lexington, KY ( 2020 ). 10.1609/aiide.v16i1.7437 bchapter

2020 doi
[32]

, Si , M

bchapter Lockwood , O. , Si , M. : Playing atari with hybrid quantum-classical reinforcement learning . In: NeurIPS 2020 Workshop on Pre-registration in Machine Learning , Virtual event ( 2021 ). http://proceedings.mlr.press/v148/lockwood21a.html bchapter

2020
[33]

, Kavukcuoglu , K

barticle Mnih , V. , Kavukcuoglu , K. , Silver , D. , Rusu , A.A. , Veness , J. , Bellemare , M.G. , Graves , A. , Riedmiller , M. , Fidjeland , A.K. , Ostrovski , G. , Petersen , S. , Beattie , C. , Sadik , A. , Antonoglou , I. , King , H. , Kumaran , D. , Wierstra , D. , Leg...

2015 doi
[34]

, Delfosse , N

barticle Nautrup , H.P. , Delfosse , N. , Dunjko , V. , Briegel , H.J. , Friis , N. : Optimizing Q uantum E rror C orrection C odes with R einforcement L earning . Quantum 3 , 215 ( 2019 ) 10.22331/q-2019-12-16-215 barticle

2019 doi
[35]

, Chuang , I.L

bbook Nielsen , M.A. , Chuang , I.L. : Quantum Computation and Quantum Information: 10th Anniversary Edition . Camb. Univ. Press , Cambridge ( 2010 ). 10.1017/CBO9780511976667 bbook

2010 doi
[36]

: A practical introduction to tensor networks: Matrix product states and projected entangled pair states

barticle Orús , R. : A practical introduction to tensor networks: Matrix product states and projected entangled pair states . Ann. Phys. 349 , 117 -- 158 ( 2014 ) 10.1016/j.aop.2014.06.013 barticle

2014 doi
[37]

, Trenkwalder , L.M

bchapter Ostaszewski , M. , Trenkwalder , L.M. , Masarczyk , W. , Scerri , E. , Dunjko , V. : Reinforcement learning for optimization of variational quantum circuit architectures . In: Advances in Neural Information Processing Systems 34: Annual Conference on Neural Informatio...

2021
[38]

, Dunjko , V

barticle Paparo , G.D. , Dunjko , V. , Makmal , A. , Martin-Delgado , M.A. , Briegel , H.J. : Quantum speedup for active learning agents . Phys. Rev. X 4 , 031002 ( 2014 ) 10.1103/PhysRevX.4.031002 barticle

2014 doi
[39]

: \"U ber eine A ufgabe der W ahrscheinlichkeitsrechnung betreffend die I rrfahrt im S tra ennetz

barticle P \'o lya , G. : \"U ber eine A ufgabe der W ahrscheinlichkeitsrechnung betreffend die I rrfahrt im S tra ennetz . Math. Ann. 84 ( 1 ), 149 -- 160 ( 1921 ) 10.1007/BF01458701 barticle

1921 doi
[40]

: Quantum C omputing in the NISQ era and beyond

barticle Preskill , J. : Quantum C omputing in the NISQ era and beyond . Quantum 2 , 79 ( 2018 ) 10.22331/q-2018-08-06-79 barticle

2018 doi
[41]

, Kreplin , D.A

botherref Rapp , F. , Kreplin , D.A. , Huber , M.F. , Roth , M. : Reinforcement learning-based architecture search for quantum machine learning (2024). https://arxiv.org/abs/2406.02717 botherref

2024 arXiv
[42]

, Laakkonen , T

botherref Ruiz , F.J.R. , Laakkonen , T. , Bausch , J. , Balog , M. , Barekatain , M. , Heras , F.J.H. , Novikov , A. , Fitzpatrick , N. , Romera-Paredes , B. , Wetering , J. , , Fawzi , A. , Meichanetzidis , K. , Kohli , P. : Quantum Circuit Optimization with AlphaTensor (202...

2024 arXiv
[43]

o mberg , T. , Schiansky , P. , Dunjko , V. , Friis , N. , Harris , N.C. , Hochberg , M. , Englund , D. , W \

barticle Saggio , V. , Asenbeck , B.E. , Hamann , A. , Str \"o mberg , T. , Schiansky , P. , Dunjko , V. , Friis , N. , Harris , N.C. , Hochberg , M. , Englund , D. , W \"o lk , S. , Briegel , H.J. , Walther , P. : Experimental quantum speed-up in reinforcement learning agents...

2021 doi
[44]

, Killoran , N

barticle Schuld , M. , Killoran , N. : Is quantum advantage the right goal for quantum machine learning? PRX Quantum 3 , 030101 ( 2022 ) 10.1103/PRXQuantum.3.030101 barticle

2022 doi
[45]

, Wolski , F

botherref Schulman , J. , Wolski , F. , Dhariwal , P. , Radford , A. , Klimov , O. : Proximal Policy Optimization Algorithms (2017). https://arxiv.org/abs/1707.06347 botherref

2017 arXiv
[46]

, Lever , G

bchapter Silver , D. , Lever , G. , Heess , N. , Degris , T. , Wierstra , D. , Riedmiller , M. : Deterministic policy gradient algorithms . In: Proceedings of the 31st International Conference on Machine Learning , Beijing, China , pp. 387 -- 395 ( 2014 ). https://proceedings....

2014
[47]

, Schrittwieser , J

barticle Silver , D. , Schrittwieser , J. , Simonyan , K. , Antonoglou , I. , Huang , A. , Guez , A. , Hubert , T. , Baker , L. , Lai , M. , Bolton , A. , Chen , Y. , Lillicrap , T. , Hui , F. , Sifre , L. , Driessche , G. , Graepel , T. , Hassabis , D. : Mastering the game of...

2017 doi
[48]

, Eickbusch , A

barticle Sivak , V.V. , Eickbusch , A. , Liu , H. , Royer , B. , Tsioutsios , I. , Devoret , M.H. : Model-free quantum control with reinforcement learning . Phys. Rev. X 12 , 011059 ( 2022 ) 10.1103/PhysRevX.12.011059 barticle

2022 doi
[49]

, Eickbusch , A

barticle Sivak , V.V. , Eickbusch , A. , Royer , B. , Singh , S. , Tsioutsios , I. , Ganjam , S. , Miano , A. , Brock , B.L. , Ding , A.Z. , Frunzio , L. , Girvin , S.M. , Schoelkopf , R.J. , Devoret , M.H. : Real-time quantum error correction beyond break-even . Nature 616 ( ...

2023 doi
[50]

, Jerbi , S

barticle Skolik , A. , Jerbi , S. , Dunjko , V. : Quantum agents in the G ym: a variational quantum algorithm for deep Q -learning . Quantum 6 , 720 ( 2022 ) 10.22331/q-2022-05-24-720 barticle

2022 doi
[51]

, W \"o lk , S

barticle Sriarunothai , T. , W \"o lk , S. , Giri , G.S. , Friis , N. , Dunjko , V. , Briegel , H.J. , Wunderlich , C. : Speeding-up the decision making of a learning agent using an ion trap quantum processor . Quantum Sci. Technol. 4 ( 1 ), 015014 ( 2018 ) 10.1088/2058-9565/a...

2018 doi
[52]

, Barto , A.G

bbook Sutton , R.S. , Barto , A.G. : Reinforcement Learning: An Introduction. 2nd Edition . A Bradford Book , Cambridge ( 2018 ). http://incompleteideas.net/book/the-book-2nd.html bbook

2018
[53]

, Dayan , P

barticle Watkins , C.J.C.H. , Dayan , P. : Q-learning . Mach. Learn. 8 ( 3 ), 279 -- 292 ( 1992 ) 10.1007/BF00992698 barticle

1992 doi
[54]

, Hein , D

botherref Wiedemann , S. , Hein , D. , Udluft , S. , Mendl , C. : Quantum Policy Iteration via Amplitude Estimation and Grover Search -- Towards Quantum Advantage for Reinforcement Learning (2023). https://arxiv.org/abs/2206.04741 botherref

2023 arXiv
[55]

, Hu , J

botherref Zhong , H. , Hu , J. , Xue , Y. , Li , T. , Wang , L. : Provably Efficient Exploration in Quantum Reinforcement Learning with Logarithmic Worst-Case Regret (2024). https://arxiv.org/abs/2302.10796 botherref

2024 arXiv
[56]

write newline

" write newline "" before.all 'output.state := FUNCTION string.to.integer 't := t text.length 'k := #1 'char.num := t char.num #1 substring 's := s is.num s "." = or char.num k = not and char.num #1 + 'char.num := while char.num #1 - 'char.num := t #1 char.num substring FUNCTI...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

, Bergholm , V

botherref Biamonte , J. , Bergholm , V. : Tensor Networks in a Nutshell (2017). https://arxiv.org/abs/1708.00006 botherref

arXiv 2017

[2] [2]

, Wittek , P

barticle Biamonte , J. , Wittek , P. , Pancotti , N. , Rebentrost , P. , Wiebe , N. , Lloyd , S. : Quantum machine learning . Nature 549 ( 7671 ), 195 -- 202 ( 2017 ) 10.1038/nature23474 barticle

doi:10.1038/nature23474 2017

[3] [3]

, Brassard , G

barticle Boyer , M. , Brassard , G. , H yer , P. , Tapp , A. : Tight bounds on quantum searching . Fortschr. Phys. 46 ( 4-5 ), 493 -- 505 ( 1998 ) 10.1002/(SICI)1521-3978(199806)46:4/5<493::AID-PROP493>3.0.CO;2-P barticle

doi:10.1002/(sici)1521-3978(199806)46:4/5 1998

[4] [4]

, H yer , P

barticle Brassard , G. , H yer , P. , Mosca , M. , Tapp , A. : Quantum amplitude amplification and estimation . Contemp. Math. 305 , 53 -- 74 ( 2002 ) 10.1090/conm/305/05215 barticle

doi:10.1090/conm/305/05215 2002

[5] [5]

, Chubb , C.T

barticle Bridgeman , J.C. , Chubb , C.T. : Hand-waving and interpretive dance: an introductory course on tensor networks . J. Phys. A 50 ( 22 ), 223001 ( 2017 ) 10.1088/1751-8121/aa6dc3 barticle

doi:10.1088/1751-8121/aa6dc3 2017

[6] [6]

, Cuevas , G

barticle Briegel , H.J. , Cuevas , G. : Projective simulation for artificial intelligence . Sci. Rep. 2 ( 1 ), 400 ( 2012 ) 10.1038/srep00400 barticle

doi:10.1038/srep00400 2012

[7] [7]

, Cheung , V

botherref Brockman , G. , Cheung , V. , Pettersson , L. , Schneider , J. , Schulman , J. , Tang , J. , Zaremba , W. : OpenAI Gym (2016). https://arxiv.org/abs/1606.01540 botherref

arXiv 2016

[8] [8]

, Day , A.G.R

barticle Bukov , M. , Day , A.G.R. , Sels , D. , Weinberg , P. , Polkovnikov , A. , Mehta , P. : Reinforcement learning in different phases of quantum control . Phys. Rev. X 8 , 031086 ( 2018 ) 10.1103/PhysRevX.8.031086 barticle

doi:10.1103/physrevx.8.031086 2018