pith. sign in

arxiv: 2504.11737 · v2 · submitted 2025-04-16 · 🪐 quant-ph

Hardware Co-Designed Optimal Control for Programmable Atomic Quantum Processors via Reinforcement Learning

Qian Ding , Dirk Englund This is my paper

Pith reviewed 2026-05-22 21:03 UTC · model grok-4.3

classification 🪐 quant-ph
keywords quantum optimal controlreinforcement learningatomic qubitsoptical crosstalkhardware co-designsingle-qubit gatesphotonic controlgate fidelity
0
0 comments X

The pith

An end-to-end differentiable reinforcement learning method finds control pulses that keep single-qubit gate fidelity above 99.9 percent even with realistic optical crosstalk and beam leakage.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper builds a mathematical model of photonic hardware imperfections and folds it directly into a quantum optimal control loop. It then trains three different optimizers on that model and shows that only the end-to-end differentiable reinforcement learning version reaches and holds gate fidelities above 99.9 percent while converging faster and staying stable when crosstalk strength or control noise changes. A reader would care because scalable atomic processors need many atoms addressed in parallel without fidelity collapsing under the real imperfections of the optical beams that control them. The work therefore treats the hardware limitations as part of the optimization problem rather than an external constraint to be ignored.

Core claim

Integrating a mathematical model of the photonic control hardware into the quantum optimal control framework and applying an end-to-end differentiable RL method enables robust, high-fidelity parallel single-qubit gate operations, consistently achieving fidelities above 99.9 percent under realistic conditions of channel crosstalk and dynamic control imperfections, with faster convergence than the SADE-Adam baseline or conventional PPO.

What carries the argument

The end-to-end differentiable reinforcement learning optimizer that back-propagates through both the quantum dynamics and the hardware crosstalk model to produce control pulses.

If this is right

  • Gate fidelity remains above 99.9 percent as the number of addressed atoms grows.
  • Performance holds across a range of fixed crosstalk strengths.
  • The method stays robust when control signals include randomized dynamic imperfections.
  • Standard PPO degrades with increasing system size while the differentiable version does not.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same co-design loop could be retrained for multi-qubit entangling gates if the hardware model is extended to include two-atom interactions.
  • Controllers trained this way might replace fixed analytic pulse shapes in future atomic processors.
  • If the hardware model is updated with measured data from a real device, the RL policy could be fine-tuned on the physical system without starting from scratch.

Load-bearing premise

The constructed mathematical model of the photonic control hardware must accurately capture the dominant real-world imperfections such as inter-channel crosstalk and beam leakage.

What would settle it

Measure actual gate fidelity on a physical atomic array driven by the pulses produced by the learned policy and compare it to the simulated 99.9 percent value under the same measured crosstalk levels; a large drop below 99 percent would falsify the claim that the method transfers.

Figures

Figures reproduced from arXiv: 2504.11737 by Dirk Englund, Qian Ding.

Figure 1
Figure 1. Figure 1: FIG. 1. (a) Workflow of the implemented hardware co-designed QOC framework. The process starts with defining system [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: FIG. 2. (a) A 16-channel programmable atomic PIC control hardware fabricated using a 200 mm CMOS process [ [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: (a), would implement Utarget with high fidelity, as the control field would not influence neighboring atoms. However, when atoms arranged in a triangular subgroup with a spacing of 3 µm and a beam waist of 2 µm, the control field applied to atom 1 inevitably leaks onto atoms 2 and 3, as illustrated in the left plots of the field’s am￾plitude and phase profiles in [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: FIG. 4. (a) Gate error optimization progress for intermediate and difficult tasks with [PITH_FULL_IMAGE:figures/full_fig_p010_4.png] view at source ↗
read the original abstract

Developing scalable, fault-tolerant atomic quantum processors requires precise control over large arrays of optical beams. This remains a major challenge due to inherent imperfections in classical control hardware, such as inter-channel crosstalk and beam leakage. In this work, we introduce a hardware co-designed intelligent quantum control framework to address these limitations. We construct a mathematical model of the photonic control hardware, integrate it into the quantum optimal control (QOC) framework, and apply reinforcement learning (RL) techniques to discover optimal control strategies. We demonstrate that the proposed framework enables robust, high-fidelity parallel single-qubit gate operations under realistic control conditions, where each atom is individually addressed by an optical beam. Specifically, we implement and benchmark three optimization strategies: a classical hybrid Self-Adaptive Differential Evolution-Adam (SADE-Adam) optimizer, a conventional RL approach based on Proximal Policy Optimization (PPO), and a novel end-to-end differentiable RL method. Using SADE-Adam as a baseline, we find that while PPO performance degrades as system complexity increases, the end-to-end differentiable RL consistently achieves gate fidelities above 99.9$\%$, exhibits faster convergence, and maintains robustness under varied channel crosstalk strength and randomized dynamic control imperfections.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript introduces a hardware co-designed framework for quantum optimal control of atomic processors. It constructs a mathematical model of photonic hardware that includes inter-channel crosstalk and beam leakage, embeds this model into the QOC problem, and applies three optimizers—SADE-Adam (baseline), PPO, and a novel end-to-end differentiable RL method—to discover pulse sequences for parallel single-qubit gates. The central claims are that the differentiable RL approach consistently reaches gate fidelities above 99.9 %, converges faster than the alternatives, and remains robust when crosstalk strength and randomized dynamic imperfections are varied inside the simulation.

Significance. If the hardware model is shown to be faithful to real devices, the work would demonstrate a practical route to co-designing control waveforms that explicitly compensate for classical imperfections, which is relevant for scaling neutral-atom arrays. The explicit integration of a differentiable hardware model into the RL loop and the head-to-head comparison of three distinct optimizers are positive features. However, the absence of any experimental grounding for the model means the reported performance numbers and robustness statements remain simulation artifacts whose transferability is unproven.

major comments (2)
  1. [§3 (Hardware Model)] §3 (Hardware Model): The mathematical model of crosstalk and leakage is introduced and inserted into the QOC cost function, yet no calibration against measured data from actual photonic beam arrays, no parameter fitting to experimental traces, and no side-by-side comparison of simulated versus observed leakage spectra are provided. Because the fidelity and robustness claims are asserted under “realistic control conditions,” this omission is load-bearing.
  2. [§5 (Numerical Results)] §5 (Numerical Results) and abstract: Gate fidelities >99.9 % and the statement that performance “maintains robustness under varied channel crosstalk strength” are reported exclusively from trajectories inside the unvalidated model. No sensitivity analysis to plausible model mismatches (e.g., time-varying crosstalk, higher-order diffraction, or nonlinear leakage) is shown; therefore the quantitative superiority over SADE-Adam and PPO cannot yet be regarded as transferable.
minor comments (2)
  1. [Abstract] The abstract states performance numbers without specifying the number of atoms, the Hilbert-space dimension, or the precise figure of merit (average gate fidelity, worst-case fidelity, etc.) used to obtain the 99.9 % threshold.
  2. [§4 (RL Methods)] Notation for the differentiable RL policy gradient is introduced without an explicit equation reference; readers must infer the back-propagation path through the hardware model from the text alone.

Simulated Author's Rebuttal

2 responses · 1 unresolved

We thank the referee for the constructive feedback and for highlighting both the strengths of the hardware-co-design approach and the need for clearer scoping of the simulation results. We address the two major comments below. Our study is a computational demonstration of the end-to-end differentiable RL framework; we therefore cannot supply new experimental calibration data at this stage.

read point-by-point responses

  1. Referee: [§3 (Hardware Model)] The mathematical model of crosstalk and leakage is introduced and inserted into the QOC cost function, yet no calibration against measured data from actual photonic beam arrays, no parameter fitting to experimental traces, and no side-by-side comparison of simulated versus observed leakage spectra are provided. Because the fidelity and robustness claims are asserted under “realistic control conditions,” this omission is load-bearing.

    Authors: We agree that the model parameters are not fitted to new experimental traces from a specific apparatus. The crosstalk and leakage coefficients are drawn from typical values reported in the neutral-atom literature (e.g., beam-waist overlap and diffraction estimates). The manuscript’s contribution is the integration of such a model into a fully differentiable RL loop and the head-to-head optimizer comparison; the numerical results therefore demonstrate performance inside the chosen model rather than direct experimental prediction. We will revise the abstract and §3 to replace “realistic control conditions” with “modeled control imperfections” and add an explicit limitations paragraph stating that experimental calibration remains future work. revision: partial

  2. Referee: [§5 (Numerical Results)] and abstract: Gate fidelities >99.9 % and the statement that performance “maintains robustness under varied channel crosstalk strength” are reported exclusively from trajectories inside the unvalidated model. No sensitivity analysis to plausible model mismatches (e.g., time-varying crosstalk, higher-order diffraction, or nonlinear leakage) is shown; therefore the quantitative superiority over SADE-Adam and PPO cannot yet be regarded as transferable.

    Authors: All reported fidelities and robustness curves are generated inside the defined model; we do not claim direct transferability to hardware. The existing figures already vary crosstalk amplitude over an order of magnitude and include randomized dynamic imperfections. We can add a supplementary sensitivity study that perturbs the model with time-varying crosstalk and higher-order diffraction terms to quantify degradation. This will be included as an additional panel and a short discussion in the revised §5, while the abstract will be updated to qualify the 99.9 % figure as “within the simulated hardware model.” revision: partial

standing simulated objections not resolved
  • Experimental calibration or side-by-side comparison of the crosstalk/leakage model against measured data from a physical photonic beam array.

Circularity Check

0 steps flagged

No circularity: results are direct simulation outputs from an independently constructed hardware model.

full rationale

The paper constructs a mathematical model of photonic hardware (crosstalk, leakage) and applies RL optimizers (SADE-Adam, PPO, end-to-end differentiable RL) inside that model to obtain gate fidelities and robustness metrics. These outputs are computed results from the model equations and RL training loops rather than quantities defined in terms of themselves or recovered by fitting the target metric. No self-citation chains, uniqueness theorems, or ansatzes imported from prior author work are referenced in the abstract or setup; the model parameters are stated as inputs, not outputs. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Review performed on abstract only; no explicit free parameters, axioms, or invented entities are identifiable from the provided text. The central claim depends on the fidelity of the un-detailed hardware model and the correctness of the RL training procedure.

pith-pipeline@v0.9.0 · 5741 in / 1245 out tokens · 42375 ms · 2026-05-22T21:03:22.488095+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 37 canonical work pages · 2 internal anchors

  1. [1]

    Preskill

    J. Preskill. Quantum computing in the nisq era and beyond.Quantum, 2:79, 2018

    work page 2018

  2. [2]

    Montanaro

    A. Montanaro. Quantum algorithms: an overview.NPJ Quantum Information, 2:15023, 2016

    work page 2016

  3. [3]

    Quantumsupremacyusingaprogrammable superconducting processor.Nature, 574:505–510, 2019

    F.Aruteetal. Quantumsupremacyusingaprogrammable superconducting processor.Nature, 574:505–510, 2019

    work page 2019

  4. [4]

    Bharti et al

    K. Bharti et al. Noisy intermediate-scale quantum algo- rithms.Reviews of Modern Physics, 94:015004, 2022

    work page 2022

  5. [5]

    Gambetta, J

    J. Gambetta, J. M. Chow, and M. Steffen. Building logical qubits in a superconducting quantum computing system. NPJ Quantum Information, 3:2, 2017

    work page 2017

  6. [6]

    Suppressing quantum errors by scaling a surface code logical qubit.Nature, 614:676–681, 2023

    Google Quantum AI. Suppressing quantum errors by scaling a surface code logical qubit.Nature, 614:676–681, 2023

    work page 2023

  7. [7]

    Debnath, N

    S. Debnath, N. Linke, C. Figgatt, et al. Demonstration of a small programmable quantum computer with atomic qubits.Nature, 536:63–66, 2016

    work page 2016

  8. [8]

    Henriet et al

    L. Henriet et al. Quantum computing with neutral atoms. Quantum, 4:327, 2020

    work page 2020

  9. [9]

    Morgado and S

    M. Morgado and S. Whitlock. Quantum simulation and computing with rydberg-interacting qubits.A VS Quan- tum Sci., 3(2):023501, 2021

    work page 2021

  10. [10]

    Bluvstein, H

    D. Bluvstein, H. Levine, G. Semeghini, et al. A quantum processor based on coherent transport of entangled atom arrays.Nature, 604:451–456, 2022

    work page 2022

  11. [11]

    Ebadi et al

    S. Ebadi et al. Quantum phases of matter on a 256-atom programmable quantum simulator.Nature, 595:227–232, 2021

    work page 2021

  12. [12]

    Bluvstein, S.J

    D. Bluvstein, S.J. Evered, A.A. Geim, et al. Logical quantum processor based on reconfigurable atom arrays. Nature, 626:58–65, 2024

    work page 2024

  13. [13]

    Khaneja, R

    N. Khaneja, R. Brockett, and S. J. Glaser. Time optimal control in spin systems.Physical Review A, 63:032308, 2001

    work page 2001

  14. [14]

    Reich, Mamadou Ndong, and Christiane P

    Daniel M. Reich, Mamadou Ndong, and Christiane P. Koch. Monotonically convergent optimization in quan- tum control using krotov’s method.J. Chem. Phys, 136(10):104103, 2012

    work page 2012

  15. [15]

    Caneva, T

    T. Caneva, T. Calarco, and S. Montangero. Chopped random-basis quantum optimization.Physical Review A, 84:022326, 2011

    work page 2011

  16. [16]

    Bukov, A

    M. Bukov, A. G. Green, and D. Sels. Reinforcement learning in different phases of quantum control.Physical Review X, 8:031086, 2018

    work page 2018

  17. [17]

    Y. Yang, Y. Liu, and Y. Wang. Reinforcement learning for quantum control: Fundamentals, methods, and recent progress.Chinese Physics B, 29(9):090308, 2020

    work page 2020

  18. [18]

    Porotti, D

    R. Porotti, D. Tamascelli, and M. G. A. Paris. Deep rein- forcementlearningforquantumoptimalcontrol.Quantum, 6:712, 2022

    work page 2022

  19. [19]

    V. V. Sivak, A. Eickbusch, H. Liu, et al. Model-free quantum control with reinforcement learning.Phys. Rev. X, 12:011059, 2022

    work page 2022

  20. [20]

    C. P. Koch, U. Boscain, T. Calarco, et al. Quantum optimal control in quantum technologies. strategic report on current status, visions and goals for research in europe. EPJ Quantum Technology, 9:19, 2022

    work page 2022

  21. [21]

    A. J. Menssen, A. Hermans, I. Christen, T. Propson, C. Li, A. J. Leenheer, M. Zimmermann, M. Dong, H. Larocque, H. Raniwala, G. Gilbert, M. Eichenfield, and D. R. En- glund. Scalable photonic integrated circuits for high- fidelity light control.Optica, 10:1366–1372, 2023

    work page 2023

  22. [22]

    Christen, T

    I. Christen, T. Propson, M. Sutula, et al. An integrated photonic engine for programmable atomic control.Nature Communications, 16:82, 2025

    work page 2025

  23. [23]

    Leenheer, Matthew Zimmermann, Daniel Dominguez, Adrian J

    Mark Dong, Genevieve Clark, Andrew J. Leenheer, Matthew Zimmermann, Daniel Dominguez, Adrian J. Menssen, David Heim, Gerald Gilbert, Dirk Englund, and Matt Eichenfield. High-speed programmable photonic cir- cuits in a cryogenically compatible, visible–near-infrared 200mm cmos architecture.Nature Photonics, 16(1):59–65, Jan 2022

    work page 2022

  24. [24]

    Eichenfield

    P.R.Stanfield, A.J.Leenheer, C.P.Michael, R.Sims, and M. Eichenfield. Cmos-compatible, piezo-optomechanically tunable photonics for visible wavelengths and cryogenic temperatures.Optics Express, 27:28588–28605, 2019

    work page 2019

  25. [25]

    Coupled-mode theory for optical waveg- uides: an overview.J

    Wei-Ping Huang. Coupled-mode theory for optical waveg- uides: an overview.J. Opt. Soc. Am. A, 11(3):963–983, Mar 1994

    work page 1994

  26. [26]

    E. T. Jaynes and F. W. Cummings. Comparison of quan- tum and semiclassical radiation theories with application to the beam maser.Proceedings of the IEEE, 51(1):89–109, 1963

    work page 1963

  27. [27]

    Levine, D

    H. Levine, D. Bluvstein, A. Keesling, et al. Dispersive optical systems for scalable raman driving of hyperfine qubits.Physical Review A, 105:032618, 2022

    work page 2022

  28. [28]

    Proximal Policy Optimization Algorithms

    John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms.ArXiv, abs/1707.06347, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  29. [29]

    JAX: compos- able transformations of Python+NumPy programs

    James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman-Milne, and Qiao Zhang. JAX: compos- able transformations of Python+NumPy programs. http://github.com/jax-ml/jax, 2018. Version 0.3.13

    work page 2018

  30. [30]

    A. K. Qin, V. L. Huang, and P. N. Suganthan. Differ- ential evolution algorithm with strategy adaptation for global numerical optimization.IEEE Transactions on Evolutionary Computation, 13(2):398–417, 2009

    work page 2009

  31. [31]

    L. Jiao, F. Liu, S. Wu, B. Hou, and X. Wang. Advances in differential evolution.Swarm and Evolutionary Com- putation, 54:100665, 2020

    work page 2020

  32. [32]

    D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. InProceedings of the International Confer- ence on Learning Representations (ICLR), 2014

    work page 2014

  33. [33]

    S. J. Reddi, S. Kale, and S. Kumar. On the convergence of adam and beyond. InProceedings of the International Conference on Learning Representations (ICLR), 2019

    work page 2019

  34. [34]

    End-to-End Robotic Reinforcement Learning without Reward Engineering

    Avi Singh, Larry Yang, Kristian Hartikainen, Chelsea Finn, and Sergey Levine. End-to-end robotic rein- forcement learning without reward engineering.ArXiv, abs/1904.07854, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1904

  35. [35]

    Deep learning.nature, 521(7553):436, 2015

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning.nature, 521(7553):436, 2015

    work page 2015

  36. [36]

    Curriculum learning

    Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. InProceedings of the 26th Annual International Conference on Machine Learning, ICML ’09, page 41–48, New York, NY, USA,

    work page

  37. [37]

    Association for Computing Machinery

    work page