Pith. sign in

REVIEW 4 major objections 5 minor 62 references

Comparing Behavioural Cloning and Reinforcement Learning for Spacecraft Guidance and Control Networks

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Reinforcement learning for spacecraft guidance networks can surpass the expert data that behavioural cloning imitates: in a 67P comet landing, the RL network uses 0.5 revolutions instead of 1.5 and arrives 4.2 hours sooner with less fuel.

desk verdict A genuinely useful first direct BC-vs-RL comparison for spacecraft G&CNETs, with a real discovery on 67P, but the headline robustness claim is confounded by unequal training distributions. read the letter →

arxiv 2507.19535 v1 pith:NQ4XV6UM submitted 2025-07-22 eess.SY astro-ph.EPastro-ph.IMcs.LGcs.SY

Harry Holt , Sebastien Origer , Dario Izzo This is my paper
classification eess.SYastro-ph.EPastro-ph.IMcs.LGcs.SY
keywords guidanceandcontrolnetworksbehaviouralcloningreinforcementlearninglow-thrusttrajectoryoptimizationspacecraftautonomyrewardredistributioncontinuous-thruststochasticrobustness
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper directly compares behavioural cloning (BC), in which a guidance-and-control network learns by imitating expert optimal trajectories, with reinforcement learning (RL), in which the network learns by trial and error against a reward function, across four spacecraft transfer and landing problems. It finds that BC-trained networks reproduce the expert solution closely on the nominal trajectory, while RL-trained networks are more robust under off-nominal conditions and can in one case beat the expert data itself. In the 67P comet-landing scenario the RL network replaces a 1.5-revolution arc with a 0.5-revolution arc, reaching the 5 m target about 4.2 hours earlier and using slightly less fuel than the locally optimal indirect-method solution that generated the BC training data. The paper also contributes an RL training recipe—evaluating the network inside the integrator and redistributing sparse rewards—so that one RL setup works for time-optimal, time-fixed fuel-optimal, and time-free fuel-optimal problems. If true, the practical upshot is that BC is the cheaper near-optimal choice when a good expert trajectory exists, while RL is the stronger choice under uncertainty and a route to better-than-expert guidance laws.

What carries the argument

The central object is the G&CNET: a small feed-forward neural network $u_{NN}(x)$ that maps the spacecraft state to a continuous thrust direction and throttle, closing the loop as $f(x)+g(x)u_{NN}(x)$. For behavioural cloning, the load-bearing mechanism is backward generation of optimal examples (BGOE): perturbing the final costates of one nominal indirect-method solution creates hundreds of thousands of optimal trajectories, sampled in 100 points and used as supervised state-action labels, which is why dataset quality and coverage set the ceiling for BC. For reinforcement learning, two additions carry the argument: evaluating the network inside the Taylor integrator so control updates happen at the integrator frequency rather than at action-sample frequency, and a reward-redistribution scheme that spreads a sparse terminal reward over all trajectory states, with a two-body-arc propellant-cost term in the fuel reward that converts position and velocity errors into fuel usage. This lets one RL setup handle time-optimal, time-fixed mass-optimal, and time-free mass-optimal problems without a pre-solved reference trajectory.

What would settle it

Retrain the behavioural-cloning network on the same randomized initial-state distribution used for the RL agent and rerun the four Monte Carlo test sets; if BC then matches RL's convergence rates, the reported robustness advantage is a training-distribution effect rather than an intrinsic RL property. Separately, run a certified global optimal-control solver on the 67P landing with the same surface constraint; if a trajectory better than the RL network's 0.5-revolution arc exists, or if the indirect method's solution is actually globally optimal, the paper's strongest claim would be overturned.

Watch

Extended reading notes

Core claim

The paper's central claim is that behavioural cloning and reinforcement learning are not just two interchangeable ways to train a guidance-and-control network; they have a sharp trade-off, and RL can in some cases do something BC cannot. On nominal trajectories BC-G&CNETs replicate the expert optimal control structure closely and often match or approach the indirect-method solution, but they inherit the expert dataset's quality and coverage limits. RL-G&CNETs give up a little nominal optimality yet remain robust when initial conditions, orbit-determination data, or thrust execution are noisy. The strongest evidence is the 67P landing: the expert dataset came from a locally optimal indirect-method solution that takes 1.5 revolutions, about 15.74 hours, to reach the 5 m target, while the RL network finds a 0.5-revolution trajectory that reaches the target in 11.56 hours, uses slightly less fuel, and closes to a lower velocity residual; closing the RL end state with the indirect method confirms the combined solution beats the original local optimum. The paper concludes that RL can reveal globally optimal strategies that eluded the generation of training samples, rather than merely mimicking a suboptimal expert.

Load-bearing premise

The load-bearing premise is that the robustness gap comes from BC versus RL as methods, not from the different data each one sees: BC is trained only on trajectories clustered around a single expert solution, while RL is trained on randomized initial states, so if BC were given the same starting-state variety its reported disadvantage could shrink or disappear.

Editorial extensions

If this is right

  • When a good expert solution exists, BC-trained G&CNETs replicate it to fractions of a percent on the nominal trajectory, so BC remains a cheap near-optimal route for deterministic, well-known transfers.
  • In the 67P landing, RL replaces a 1.5-revolution arc with a 0.5-revolution arc, arriving about 4.2 hours sooner and using slightly less fuel while meeting the 5 m position constraint, so RL can improve on a locally optimal expert solution.
  • Across the Monte Carlo tests for initial-condition, orbit-determination, missed-thrust, and thrust-execution errors, RL keeps full-state convergence rates between 74.5% and 100%, while BC drops to 0-22% in the hardest scenarios, supporting RL for uncertain operations.
  • The reward-redistribution and integrator-frequency mechanisms remove the need for a pre-solved reference trajectory or a hand-crafted dense reward, so the same RL recipe applies to time-optimal and fuel-optimal problems.
  • BC remains far more sample-efficient, using tens of millions of samples versus up to 634 million for RL in the GTOC 11 case, so the choice between the two methods is an operational trade-off rather than a one-sided verdict.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: a direct extension the paper does not run is to train BC on the same randomized initial-state distribution used for RL and add noise-based augmentation; this would isolate whether RL's robustness edge is intrinsic or merely inherited from wider training coverage.
  • Editorial inference: the 67P result implies a practical closed loop the paper only hints at—use RL as an initial-guess generator for indirect optimal-control solvers, then use the improved solutions to build better BC datasets, combining RL discovery with BC sample efficiency.
  • Editorial inference: if RL can switch control topologies such as 0.5 versus 1.5 revolutions as the reward geometry changes, then validation of neural guidance networks should examine which class of trajectories the network encodes, not only its pointwise approximation error against one expert solution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper compares behavioural cloning (BC) and reinforcement learning (RL) for training G&CNETs on four continuous-thrust spacecraft transfer problems: GTOC 11, Earth-Mars, Psyche, and 67P. The authors introduce an RL training framework that integrates the G&CNET control inside the Taylor integrator, decoupling action and control frequencies, and a reward redistribution scheme for sparse terminal rewards. Nominal performance is evaluated against indirect-method optimal solutions, and robustness is tested with Monte Carlo simulations under initial-condition, orbit-determination, zero-order-hold, and execution errors. The main claims are that BC replicates expert optimal behaviour well on nominal conditions, while RL is more robust to stochastic conditions and, in the 67P landing, discovers a 0.5-revolution solution that improves on the local-optimal expert trajectory used to generate the BC dataset.

Significance. If the claims are substantiated, this is a useful and timely comparison that addresses a gap in the G&CNET literature. The breadth of scenarios is a strength, and the 67P result is given a valuable independent check by closing the RL final state with the indirect method. The proposed RL framework, particularly the reward redistribution and integrated-control formulation, is clearly described and potentially reusable. However, the central robustness comparison is currently confounded by differences in training distributions and network architectures, and the global-optimality claim exceeds the evidence. These issues are fixable but require additional experiments or careful reframing before the paper's headline conclusions can be accepted.

major comments (4)
  1. [Section IV.C, Tables 11-12] The central robustness claim is confounded by an asymmetry in training distributions. BC expert data are generated exclusively by BGOE, i.e., by perturbing final costates around one nominal solution (Section III.B), whereas the RL policy is trained with stochastic initial conditions, as stated in Section IV.C.1. Consequently, the convergence percentages in Tables 11-12 may reflect whether the tested perturbations were present in the training distribution rather than an intrinsic adaptability advantage of RL over BC. The paper itself notes the remedy: 'Future work can look to improve BC performance by adding trajectories to the database of expert examples.' To support the abstract's 'superior adaptability to stochastic conditions' claim, add a BC ablation trained on expert data covering the same stochastic IC/OD/EX distributions used in evaluation, or an RL baseline trained on nominal initial conditions only, and report Tables 11-12 for those variants.
  2. [Section III.A, Figures 1-2] The BC and RL G&CNETs do not share the same architecture: BC uses periodic SIREN activations while RL uses Softplus/Tanh activations. The paper explains this choice by citing prior results, but the nominal-performance differences in Tables 8-9, such as BC being 0.30% from optimal on GTOC 11 versus RL at 0.91%, are therefore not attributable solely to the training paradigm. A cross-condition control, such as training BC with Softplus or RL with SIREN, or at least an explicit discussion of how activation-function differences might mediate the results, is needed before the general conclusion that BC-trained G&CNETs excel at replicating expert behaviour can be drawn.
  3. [Abstract and Section IV.B.2, Table 9] The phrase 'sometimes revealing globally optimal strategies that eluded the generation of training samples' overstates the 67P result. The paper states that the indirect-method solution is a local optimum and that 'a more rigorous search for the true optimal solution would, of course, improve upon the RL-G&CNET solution.' The supported claim is that RL improved on a local-optimal expert demonstration, not that it found a globally optimal strategy. Please revise the abstract and Section V accordingly and qualify the 67P result as an improvement over a local optimum.
  4. [Table 8, Earth-Mars row] The Earth-Mars BC row reports an 'Optimality Residual' of -2.29% despite a velocity residual of 1501.55 m/s, which exceeds the convergence threshold c_v = 1000 m/s given in Table 2. Because the compared trajectories terminate at the SOI event rather than at the target state, a higher mass ratio at that event is not evidence of superior fuel optimality; the BC trajectory has not achieved the target state. Please either close the velocity residual by adding the remaining required delta-v, or explicitly report the comparison as event-level only and remove the fuel-savings interpretation.
minor comments (5)
  1. [Section II.B.1 and Section IV.B.1] The text says case A (GTOC 11) uses the rotating reference frame in Section II.B.1, but Section IV.B.1 and Table 8 describe it as computed in an inertial frame; please reconcile this inconsistency.
  2. [Table 9] The Psyche row is labelled 'Time Optimal' in the Objective column, but Table 1 and the surrounding text classify Psyche as a fuel-optimal case; please correct the label.
  3. [Section III.B] The hyperparameter sentence says 'weight decay values of 2.5e-5, 2.5e-5, and 0.0 respectively, and training epochs of 500, 500, and 200 respectively' without specifying which of the scenario groups each value refers to; please make the mapping explicit.
  4. [Section III.C.3, Eq. (12)] In the redistribution schematic, the entries 'δx_{N+D}' and 'δa_{N+D}' appear to be typos for the state x_{N+D} and action a_{N+D}; in addition, the 'Returns' line is not fully explained and should define how the truncated and redistributed rewards combine.
  5. [Section IV.C] The stochastic evaluation says the same stochastic seed is used to compare BC and RL, but no seed value or reproducibility statement is provided; please include the seed or a reproducibility note.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the BC-vs-RL comparison is anchored to independently computed optimal control solutions and shared Monte Carlo seeds; the unequal training distributions are a fairness confound, not a definitional reduction.

full rationale

The paper's central claims are empirical comparisons evaluated against independently computed TPBVP/indirect-method solutions (Tables 8-9) and against common Monte Carlo perturbations (Tables 10-12). The 67P claim that the RL network beats a local-optimal expert is cross-checked by closing the RL final state with the indirect method, so the conclusion does not reduce to the training reward or the expert dataset. The RL reward shaping (Lambert-arc delta-v, alpha_L=0.1) is a training aid and is not used in validation or inference, so the reported trajectories are not forced by the reward by construction. The paper is candid that RL robustness is partly due to training with stochastic ICs ('During training, the RL-G&CNET is subject to stochastic ICs... It will therefore trade optimality to achieve this higher level of robustness'), which is a training-distribution mismatch with the BC baseline rather than a circular argument. Several methodological components cite the authors' prior work (BGOE, SIREN activation choice, event detection), but these citations are not used to establish the central comparative result and do not smuggle in the conclusion. The overstated phrase 'globally optimal strategies' in the abstract is contradicted by the authors' own caveat that a more rigorous search would improve on the RL solution, but this is an over-claim, not circularity.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claims rest on several hand-chosen training and reward parameters (notably alpha_L and scenario time steps), on standard astrodynamics modeling assumptions, and on the fidelity of the expert and NN-event datasets. No new physical entities are introduced.

free parameters (6)
  • Lambert arc scaling alpha_L = 0.1
    Scales the Lambert arc duration in the fuel-optimal RL reward (Section III.C.2); hand-chosen after a grid search and directly shapes the reward that produced the RL results.
  • Lambert arc duration delta_t_L = grid-searched, alpha_L * (c_v T_max / m_i)
    Unknown parameter in reward function (Eq. 10); authors state a small grid search is needed to encourage convergence, so the final value is selected to make training work.
  • RL initial policy stochasticity sigma_0 = 0.1
    PPO exploration noise at start of training (Table 6), chosen by hand.
  • RL scenario time-step delta_t = 30 days, 8.71975 days, 0.025 rev, 0.025 rev
    Per-scenario step size in episode rollouts (Table 6), chosen by hand; affects the temporal resolution of the RL reward and control.
  • PPO hyperparameters (learning rate, clip, batch, epochs) = 3e-4, 0.2, 25, 10
    Standard PPO hyperparameters from Stable Baselines3, chosen without systematic tuning; the reported results depend on them.
  • BC dataset costate perturbation magnitudes = not reported
    BGOE bundles are generated with varying perturbation magnitudes (Section III.B), but numeric values are not given; the dataset distribution is a hand-chosen input that determines BC performance.
assumptions (6)
  • domain assumption The two-body point-mass dynamics of Eqs. (1) and (2) adequately represent the spacecraft environment for the comparison.
    The G&CNETs are trained and evaluated under these dynamics; any unmodeled perturbations are excluded from the benchmark.
  • domain assumption The BGOE-generated expert trajectories used for BC are optimal (or sufficiently close to optimal) solutions of the optimal control problem.
    BC inherits the quality of the expert dataset; the authors note the 67P expert is only a local optimum, which is precisely why RL can outperform it.
  • domain assumption Stable Baselines3 PPO with the chosen hyperparameters converges to a policy that adequately maximizes the designed reward.
    The RL results assume the training runs found good policies; no convergence guarantees or multiple-seed statistics are provided.
  • domain assumption The NN-event boundary faithfully represents the asteroid or comet surface for event detection in the landing scenarios.
    Termination of landing trajectories is governed by a learned event manifold from prior work [48,51]; the results depend on its accuracy.
  • ad hoc to paper The Lambert arc used inside the fuel-optimal reward can be ignored at inference, and its delta-v is a meaningful surrogate for remaining required control effort.
    This is a training-only device introduced in Section III.C.2; the paper provides no proof that it does not bias the learned policy away from the true optimal.
  • standard math Optimal control solutions used as ground truth are computed with Pontryagin's maximum principle via indirect methods.
    The optimal trajectories for BC and the reference optimality metrics rely on indirect TPBVP solutions; this is a standard tool in the field.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Comparing Behavioural Cloning and Reinforcement Learning for Spacecraft Guidance and Control Networks." pith.science (2026). https://pith.science/paper/NQ4XV6UM

@misc{pith2026250719535,
  author       = {Pith},
  title        = {Pith review of: Comparing Behavioural Cloning and Reinforcement Learning for Spacecraft Guidance and Control Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NQ4XV6UM}},
  note         = {Machine review of arXiv:2507.19535}
}
read the original abstract

Guidance & control networks (G&CNETs) provide a promising alternative to on-board guidance and control (G&C) architectures for spacecraft, offering a differentiable, end-to-end representation of the guidance and control architecture. When training G&CNETs, two predominant paradigms emerge: behavioural cloning (BC), which mimics optimal trajectories, and reinforcement learning (RL), which learns optimal behaviour through trials and errors. Although both approaches have been adopted in G&CNET related literature, direct comparisons are notably absent. To address this, we conduct a systematic evaluation of BC and RL specifically for training G&CNETs on continuous-thrust spacecraft trajectory optimisation tasks. We introduce a novel RL training framework tailored to G&CNETs, incorporating decoupled action and control frequencies alongside reward redistribution strategies to stabilise training and to provide a fair comparison. Our results show that BC-trained G&CNETs excel at closely replicating expert policy behaviour, and thus the optimal control structure of a deterministic environment, but can be negatively constrained by the quality and coverage of the training dataset. In contrast RL-trained G&CNETs, beyond demonstrating a superior adaptability to stochastic conditions, can also discover solutions that improve upon suboptimal expert demonstrations, sometimes revealing globally optimal strategies that eluded the generation of training samples.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 40 canonical work pages

  1. [1]

    SMART-1: The First Time of Europe to the Moon; Wandering in the Earth – MoonSpace,

    Racca, G. D., Foing, B. H., and Coradini, M., “SMART-1: The First Time of Europe to the Moon; Wandering in the Earth – MoonSpace,” Earth,Moon,andPlanets ,Vol.85,1999,pp.379–390. https://doi.org/https://doi.org/10.1023/A:1017065326516

    work page doi:10.1023/a:1017065326516 1999

  2. [2]

    Missiondesignfordeepspace1: Alow-thrusttechnology validation mission,

    Rayman,M.D.,Chadbourne,P.A.,Culwell,J.S.,andWilliams,S.N.,“Missiondesignfordeepspace1: Alow-thrusttechnology validation mission,”Acta Astronautica, Vol. 45, No. 4, 1999, pp. 381–388. https://doi.org/https://doi.org/10.1016/S0094- 5765(99)00157-5, URL https://www.sciencedirect.com/science/article/pii/S0094576599001575, third IAA International Conference on...

    doi:10.1016/s0094- 1999

  3. [3]

    The Dawn Spacecraft,

    Thomas, V. C., Makowski, J. M., Brown, G. M., McCarthy, J. F., Bruno, D., Cardoso, J. C., Chiville, W. M., Meyer, T. F., Nelson, K. E., Pavri, B. E., Termohlen, D. A., Violet, M. D., and Williams, J. B., “The Dawn Spacecraft,”The Dawn Mission to Minor Planets 4 Vesta and 1 Ceres, edited by C. Russell and C. Raymond, Springer New York, New York, NY, 2012, ...

    work page doi:10.1007/978-1-4614-4903-4 2012

  4. [4]

    K.,The Ion Engines Cruise Operation and the Earth Swingby of ’Hayabusa’ (MUSES-C), 2012

    Kawaguchi, J., Fujiwara, A., and Uesugi, T. K.,The Ion Engines Cruise Operation and the Earth Swingby of ’Hayabusa’ (MUSES-C), 2012. https://doi.org/10.2514/6.IAC-04-Q.5.02

    work page doi:10.2514/6.iac-04-q.5.02 2012

  5. [5]

    System design of the hayabusa 2-asteroid sample return mission to 1999 JU3,

    Tsuda, Y., Yoshikawa, M., Abe, M., Minamino, H., and Nakazawa, S., “System design of the hayabusa 2-asteroid sample return mission to 1999 JU3,”Acta Astronautica, Vol. 91, 2013, pp. 356–362. https://doi.org/10.1016/j.actaastro.2013.06.028

    doi:10.1016/j.actaastro.2013.06.028 1999

  6. [6]

    BepiColombo-Comprehensive exploration of Mercury: Mission overview and science goals,

    Benkhoff, J., van Casteren, J., Hayakawa, H., Fujimoto, M., Laakso, H., Novara, M., Ferri, P., Middleton, H. R., and Ziethe, R., “BepiColombo-Comprehensive exploration of Mercury: Mission overview and science goals,”Planetary and Space Science, Vol. 58, No. 1-2, 2010, pp. 2–20. https://doi.org/10.1016/j.pss.2009.09.020

    doi:10.1016/j.pss.2009.09.020 2010

  7. [7]

    Real-Time Optimal Control via Deep Neural Networks: Study on Landing Problems,

    Sánchez-Sánchez, C., and Izzo, D., “Real-Time Optimal Control via Deep Neural Networks: Study on Landing Problems,” Journal of Guidance, Control, and Dynamics, Vol. 41, No. 5, 2018, pp. 1122–1135. https://doi.org/10.2514/1.G002357, URL https://arc.aiaa.org/doi/10.2514/1.G002357

    doi:10.2514/1.g002357 2018

  8. [8]

    Optimality principles in spacecraft neural guidance and control,

    Izzo, D., Blazquez, E., Ferede, R., Origer, S., Wagter, C. D., and de Croon, G. C. H. E., “Optimality principles in spacecraft neural guidance and control,”Science Robotics, Vol. 9, No. 91, 2024, p. eadi6421. https://doi.org/10.1126/scirobotics.adi6421, URL https://www.science.org/doi/abs/10.1126/scirobotics.adi6421

    doi:10.1126/scirobotics.adi6421 2024

Show all 62 references
  1. [9]

    Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning,

    Foster, D. J., Block, A., and Misra, D., “Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning,” , Nov. 2024. https://doi.org/10.48550/arXiv.2407.15007, URL http://arxiv.org/abs/2407.15007, arXiv:2407.15007 [cs]

    2024 doi arXiv

  2. [10]

    Real-Time Guidance for Low-Thrust Transfers Using Deep Neural Networks,

    Izzo, D., and Öztürk, E., “Real-Time Guidance for Low-Thrust Transfers Using Deep Neural Networks,”Journal of Guidance, Control, and Dynamics, Vol. 44, No. 2, 2021, pp. 315–327. https://doi.org/10.2514/1.G005254, URL https: //arc.aiaa.org/doi/10.2514/1.G005254. 24 Table 13 Sum...

    2021 doi

  3. [11]

    Fuel-optimal guidance using costate supervised learning with local refinement,

    Evans, A., Armellin, R., Holt, H., and Pirovano, L., “Fuel-optimal guidance using costate supervised learning with local refinement,”Acta Astronautica, Vol. 228, 2025, pp. 17–29

    2025

  4. [12]

    Real-Time Optimal Control for Spacecraft Orbit Transfer via Multiscale Deep Neural Networks,

    Cheng, L., Wang, Z., Jiang, F., and Zhou, C., “Real-Time Optimal Control for Spacecraft Orbit Transfer via Multiscale Deep Neural Networks,”IEEE Transactions on Aerospace and Electronic Systems, Vol. 55, No. 5, 2019, pp. 2436–2450. https://doi.org/10.1109/TAES.2018.2889571, UR...

    2019

  5. [13]

    Neural representation of a time optimal, constant acceleration rendezvous,

    Izzo, D., and Origer, S., “Neural representation of a time optimal, constant acceleration rendezvous,”Acta Astronautica, Vol. 204, 2023, pp. 510–517. https://doi.org/https://doi.org/10.1016/j.actaastro.2022.08.045

    2023 doi

  6. [14]

    Metric to evaluate distribution shift from behavioral cloning for fuel-optimal landing policies,

    Mulekar, O. S., Bevilacqua, R., and Cho, H., “Metric to evaluate distribution shift from behavioral cloning for fuel-optimal landing policies,”Acta Astronautica, Vol. 203, 2023, pp. 421–428

    2023

  7. [15]

    Real-time optimal control for irregular asteroid landings using deep neural networks,

    Cheng, L., Wang, Z., Song, Y., and Jiang, F., “Real-time optimal control for irregular asteroid landings using deep neural networks,”Acta Astronautica, Vol. 170, 2020, pp. 66–79. https://doi.org/https://doi.org/10.1016/j.actaastro.2019.11.039, URL https://www.sciencedirect.com...

    2020 doi

  8. [16]

    Guidance and Control Networks with Periodic Activation Functions,

    Origer, S., and Izzo, D., “Guidance and Control Networks with Periodic Activation Functions,” , May 2024. URL http://arxiv.org/abs/2405.18084, arXiv:2405.18084 [cs]

    2024 arXiv

  9. [17]

    A deep learning-based approach to real-time trajectory optimization for hypersonic vehicles,

    Shi, Y., and Wang, Z., “A deep learning-based approach to real-time trajectory optimization for hypersonic vehicles,”AIAA SciTech 2020 forum, 2020, p. 0023

    2020

  10. [18]

    Publication Title: MIT Press

    Sutton, R., and Barto, A.,Reinforcment Learning: An Introduction, MIT Press, Cambridge, 1998. Publication Title: MIT Press

    1998

  11. [19]

    Guided Policy Search,

    Levine, S., and Koltun, V., “Guided Policy Search,”Proceedings of the 30th International Conference on Machine Learning, 25 Vol. 28, PMLR, 2013, pp. 1–9. URL http://proceedings.mlr.press/v28/levine13.html, series Title: Proceedings of Machine Learning Research Issue: 3

    2013

  12. [20]

    Six Degree-of-Freedom Hovering using LIDAR Altimetry via Reinforcement Meta-Learning,

    Gaudet, B., Linares, R., and Furfaro, R., “Six Degree-of-Freedom Hovering using LIDAR Altimetry via Reinforcement Meta-Learning,” 2019, pp. 1–15. URL http://arxiv.org/abs/1911.08553, arXiv: 1911.08553

    2019 arXiv

  13. [21]

    Mastering the game of Go without human knowledge,

    Silver, D., Schrittwieser, J., Simonyan, K., Antonoglou, I., Huang, A., Guez, A., Hubert, T., Baker, L., Lai, M., Bolton, A., Chen, Y., Lillicrap, T., Hui, F., Sifre, L., van den Driessche, G., Graepel, T., and Hassabis, D., “Mastering the game of Go without human knowledge,”N...

    2017 doi

  14. [22]

    Benchmarking Reinforcement Learning Algorithms on Real-World Robots,

    Mahmood, A. R., Korenkevych, D., Vasan, G., Ma, W., and Bergstra, J., “Benchmarking Reinforcement Learning Algorithms on Real-World Robots,” , No. CoRL, 2018, pp. 1–31

    2018

  15. [23]

    Deepreinforcementlearningframeworkforautonomousdriving,

    ElSallab,A.,Abdou,M.,Perot,E.,andYogamani,S.,“Deepreinforcementlearningframeworkforautonomousdriving,” ISand TInternationalSymposiumonElectronicImagingScienceandTechnology ,2017,pp.70–76. https://doi.org/10.2352/ISSN.2470- 1173.2017.19.AVM-023

    2017

  16. [24]

    A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Moving Platform,

    Rodriguez-Ramos, A., Sampedro, C., Bavle, H., de la Puente, P., and Campoy, P., “A Deep Reinforcement Learning Strategy for UAV Autonomous Landing on a Moving Platform,”Journal of Intelligent and Robotic Systems: Theory and Applications, Vol. 93, No. 1-2, 2019, pp. 351–366. ht...

    2019 doi

  17. [25]

    Adaptive Deep Learning for High-Dimensional Hamilton-Jacobi-Bellman Equations,

    Nakamura-Zimmerer, T., Gong, Q., and Kang, W., “Adaptive Deep Learning for High-Dimensional Hamilton-Jacobi-Bellman Equations,” SIAM Journal on Scientific Computing, Vol. 43, No. 2, 2021, pp. A1221–A1247. https://doi.org/10.1137/ 19M1288802, URL http://arxiv.org/abs/1907.05317...

    2021 arXiv

  18. [26]

    A Survey on Artificial Intelligence Trends in Spacecraft Guidance Dynamics and Control,

    Izzo, D., Märtens, M., and Pan, B., “A Survey on Artificial Intelligence Trends in Spacecraft Guidance Dynamics and Control,” arXiv preprint arXiv:1812.02948, 2018

    2018 arXiv

  19. [27]

    Low-Thrust Optimal Control Via Reinforcement Learning,

    Miller, D., and Linares, R., “Low-Thrust Optimal Control Via Reinforcement Learning,”AAS, 2019, pp. 1–20. Issue: February

    2019

  20. [28]

    Guidance for Closed-Loop Transfers using Reinforcement Learning with Application to Libration Point Orbits,

    LaFarge, N. B., Miller, D., Howell, K. C., and Linares, R., “Guidance for Closed-Loop Transfers using Reinforcement Learning with Application to Libration Point Orbits,”AIAA Scitech 2020 Forum, ????

    2020

  21. [29]

    ExplorationofLongTime-of-FlightThree-BodyTransfersUsingDeepReinforcement Learning,

    Yanagida,K.,Ozaki,N.,andFunase,R.,“ExplorationofLongTime-of-FlightThree-BodyTransfersUsingDeepReinforcement Learning,” AIAA Scitech 2020 Forum, American Institute of Aeronautics and Astronautics, Orlando, FL, 2020. https: //doi.org/10.2514/6.2020-0460, URL https://arc.aiaa.org...

    2020 doi

  22. [30]

    Using Reinforcement Learning to Design a Low-Thrust Approach into a Periodic Orbit in a Multi-Body System,

    Sullivan, C. J., and Bosanac, N., “Using Reinforcement Learning to Design a Low-Thrust Approach into a Periodic Orbit in a Multi-Body System,”AIAA Scitech 2020 Forum, 2020. 26

    2020

  23. [31]

    Autonomous guidance for cislunar orbit transfers via reinforcement learning,

    Federici, L., Scorsoglio, A., Zavoli, A., Furfaro, R., et al., “Autonomous guidance for cislunar orbit transfers via reinforcement learning,”AAS/AIAA Astrodynamics Specialist Conference, American Astronautical Society Big Sky, Montana (Virtual), 2021

    2021

  24. [32]

    Reinforcement Learning for Reconfiguration Maneuver Design in Multi-Body Systems,

    Bosanac, N., Bonasera, S., Sullivan, C. J., Mcmahon, J., and Ahmed, N., “Reinforcement Learning for Reconfiguration Maneuver Design in Multi-Body Systems,”AAS Astrodynamics Specialist Conference, 2021, pp. 1–20

    2021

  25. [33]

    Adaptive ZEM/ZEV feedback guidance for rendezvous in lunar NRO with collision avoidance,

    Scorsoglio, A., “Adaptive ZEM/ZEV feedback guidance for rendezvous in lunar NRO with collision avoidance,” Ph.D. thesis, Politecnico Di Milano, University of Arizona, 2018. Issue: July

    2018

  26. [34]

    DeepLearningTechniquesforAutonomousSpacecraftGuidanceDuringProximity Operations,

    Federici,L.,Benedikter,B.,andZavoli,A.,“DeepLearningTechniquesforAutonomousSpacecraftGuidanceDuringProximity Operations,”Journal of Spacecraft and Rockets, Vol. 58, No. 6, 2021, pp. 1774–1785. https://doi.org/10.2514/1.A35076, URL https://arc.aiaa.org/doi/10.2514/1.A35076

    2021 doi

  27. [35]

    Adaptive generalized ZEM-ZEV feedback guidance for planetary landing via a deep reinforcement learning approach,

    Furfaro, R., Scorsoglio, A., Linares, R., and Massari, M., “Adaptive generalized ZEM-ZEV feedback guidance for planetary landing via a deep reinforcement learning approach,”Acta Astronautica, Vol. 171, 2020, pp. 156–171. Publisher: Elsevier Ltd

    2020

  28. [36]

    Deep reinforcement learning for six degree-of-freedom planetary landing,

    Gaudet, B., Linares, R., and Furfaro, R., “Deep reinforcement learning for six degree-of-freedom planetary landing,”Advances in Space Research, Vol. 65, No. 7, 2020, pp. 1723–1741. https://doi.org/https://doi.org/10.1016/j.asr.2019.12.030, URL https://www.sciencedirect.com/sci...

    2020 doi

  29. [37]

    Reinforcement Learning for Robust Trajectory Design of Interplanetary Missions,

    Zavoli, A., and Federici, L., “Reinforcement Learning for Robust Trajectory Design of Interplanetary Missions,”Journal of Guidance, Control, and Dynamics, Vol. 44, No. 8, 2021, pp. 1440–1453. https://doi.org/10.2514/1.G005794, URL https://doi.org/10.2514/1.G005794

    2021 doi

  30. [38]

    Densely rewarded reinforcement learning for robust low-thrust trajectory optimization,

    Hu, J., Yang, H., Li, S., and Zhao, Y., “Densely rewarded reinforcement learning for robust low-thrust trajectory optimization,” Advances in Space Research, Vol. 72, No. 4, 2023, pp. 964–981. https://doi.org/10.1016/j.asr.2023.03.050, URL https: //linkinghub.elsevier.com/retri...

    2023 doi

  31. [39]

    Robustsolarsailtrajectoriesusingproximalpolicyoptimization,

    Bianchi,C.,Niccolai,L.,andMengali,G.,“Robustsolarsailtrajectoriesusingproximalpolicyoptimization,” ActaAstronautica, Vol. 226, 2025, pp. 702–715. https://doi.org/10.1016/j.actaastro.2024.10.065, URL https://linkinghub.elsevier.com/retrieve/pii/ S0094576524006398

    2025 doi

  32. [40]

    AAS 21-315 Autonomous Guidance for multi-revolution low-thrust orbit transfer via Reinforcement Learning,

    Kwon, H., Oghim, S., and Bang, H., “AAS 21-315 Autonomous Guidance for multi-revolution low-thrust orbit transfer via Reinforcement Learning,”AAS, 2021, pp. 1–16

    2021

  33. [41]

    Optimal Q-laws via reinforcement learning with guaranteed stability,

    Holt, H., Armellin, R., Baresi, N., Hashida, Y., Turconi, A., Scorsoglio, A., and Furfaro, R., “Optimal Q-laws via reinforcement learning with guaranteed stability,”Acta Astronautica, Vol. 187, 2021, pp. 511–528. https://doi.org/10.1016/j.actaastro.2021.07. 010, URL https://li...

    2021 doi

  34. [42]

    Reinforced Lyapunov controllers for low-thrust lunar transfers,

    Holt, H., Baresi, N., and Armellin, R., “Reinforced Lyapunov controllers for low-thrust lunar transfers,”Astrodynamics, Vol. 8, No. 4, 2024, pp. 633–656. 27

    2024

  35. [43]

    When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?

    Kumar, A., Hong, J., Singh, A., and Levine, S., “When Should We Prefer Offline Reinforcement Learning Over Behavioral Cloning?” , Apr. 2022. https://doi.org/10.48550/arXiv.2204.05618, URL http://arxiv.org/abs/2204.05618, arXiv:2204.05618 [cs]

    2022 doi arXiv

  36. [44]

    Champion-level drone racing using deep reinforcement learning,

    Kaufmann, E., Bauersfeld, L., Loquercio, A., Müller, M., Koltun, V., and Scaramuzza, D., “Champion-level drone racing using deep reinforcement learning,”Nature, Vol. 620, No. 7976, 2023, pp. 982–987

    2023

  37. [45]

    End-to-end Reinforcement Learning for Time-Optimal Quadcopter Flight,

    Ferede, R., Wagter, C. D., Izzo, D., and Croon, G. C. H. E. D., “End-to-end Reinforcement Learning for Time-Optimal Quadcopter Flight,”2024 IEEE International Conference on Robotics and Automation, ICRA 2024, IEEE, United States, 2024, p. 6172–6177. https://doi.org/10.1109/ICR...

    2024

  38. [46]

    End-to-end neural network based optimal quadcopter control,

    Ferede, R., Croon, G., Wagter, C. D., and Izzo, D., “End-to-end neural network based optimal quadcopter control,”Robotics and Autonomous Systems, Vol. 172, 2024. https://doi.org/10.1016/j.robot.2023.104588, URL https://research.tudelft.nl/en/ publications/end-to-end-neural-net...

    2024

  39. [47]

    Closing the gap: Optimizing Guidance and Control Networks through Neural ODEs,

    Origer, S., and Izzo, D., “Closing the gap: Optimizing Guidance and Control Networks through Neural ODEs,” , 2024. URL https://arxiv.org/abs/2404.16908

    2024 arXiv

  40. [48]

    High-order expansion of Neural Ordinary Differential Equations flows,

    Izzo, D., Origer, S., Acciarini, G., and Biscani, F., “High-order expansion of Neural Ordinary Differential Equations flows,” arXiv preprint arXiv:2504.08769, 2025

    2025 arXiv

  41. [49]

    Revisiting high-order Taylor methods for astrodynamics and celestial mechanics,

    Biscani, F., and Izzo, D., “Revisiting high-order Taylor methods for astrodynamics and celestial mechanics,”Monthly Notices of the Royal Astronomical Society, Vol. 504, No. 2, 2021, pp. 2614–2628. https://doi.org/10.1093/mnras/stab1032, URL https://doi.org/10.1093/mnras/stab1032

    2021 doi

  42. [50]

    Reliable event detection for Taylor methods in astrodynamics,

    Biscani, F., and Izzo, D., “Reliable event detection for Taylor methods in astrodynamics,”Monthly Notices of the Royal Astronomical Society, Vol. 513, No. 4, 2022, pp. 4833–4844. https://doi.org/10.1093/mnras/stac1092, URL https://academic. oup.com/mnras/article/513/4/4833/6573873

    2022 doi

  43. [51]

    Certifying Guidance & Control Networks: Uncertainty Propagation to an Event Manifold,

    Origer, S., Izzo, D., Acciarini, G., Biscani, F., Mastroianni, R., Bannach, M., and Holt, H., “Certifying Guidance & Control Networks: Uncertainty Propagation to an Event Manifold,” , 2024. URL https://arxiv.org/abs/2410.03729

    2024 arXiv

  44. [52]

    Implicit Neural Representations with Periodic Activation Functions,

    Sitzmann, V., Martel, J. N., Bergman, A. W., Lindell, D. B., and Wetzstein, G., “Implicit Neural Representations with Periodic Activation Functions,”Proc. NeurIPS, 2020

    2020

  45. [53]

    Frequency and Generalisation of Periodic Activation Functions in Reinforcement Learning,

    Mavor-Parker, A. N., Sargent, M. J., Barry, C., Griffin, L., and Lyle, C., “Frequency and Generalisation of Periodic Activation Functions in Reinforcement Learning,” , Jul. 2024. URL http://arxiv.org/abs/2407.06756, arXiv:2407.06756 [cs]

    2024 arXiv

  46. [54]

    Adam: A method for stochastic optimization,

    Kingma, D. P., and Ba, J., “Adam: A method for stochastic optimization,”arXiv preprint arXiv:1412.6980, 2014. 28

    2014 arXiv

  47. [55]

    A unifying view on dataset shift in classification,

    Moreno-Torres, J. G., Raeder, T., Alaiz-Rodríguez, R., Chawla, N. V., and Herrera, F., “A unifying view on dataset shift in classification,”Pattern Recognition, Vol. 45, No. 1, 2012, pp. 521–530. https://doi.org/10.1016/j.patcog.2011.06.019, URL https://linkinghub.elsevier.com...

    2012 doi

  48. [56]

    DART: Noise Injection for Robust Imitation Learning,

    Laskey, M., Lee, J., Fox, R., Dragan, A., and Goldberg, K., “DART: Noise Injection for Robust Imitation Learning,” 2017. URL https://arxiv.org/abs/1703.09327

    2017 arXiv

  49. [57]

    DistillingPrivilegedInformationforDubinsTravelingSalesman Problems with Neighborhoods,

    Shin,M.K.,Park,S.-J.,Ryu,S.-K.,Kim,H.,andChoi,H.-L.,“DistillingPrivilegedInformationforDubinsTravelingSalesman Problems with Neighborhoods,” , 2024. URL https://arxiv.org/abs/2404.16721

    2024

  50. [58]

    Proximal policy optimization algorithms,

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O., “Proximal policy optimization algorithms,”arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  51. [59]

    Stable-Baselines3: Reliable Reinforcement Learning Implementations,

    Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., and Dormann, N., “Stable-Baselines3: Reliable Reinforcement Learning Implementations,”Journal of Machine Learning Research, Vol. 22, No. 268, 2021, pp. 1–8. URL http://jmlr.org/ papers/v22/20-1364.html

    2021

  52. [60]

    Comparative analysis of reinforcement learning algorithms for robust interplanetary trajectorydesign,

    Federici, L., Zavoli, A., and Furfaro, R., “Comparative analysis of reinforcement learning algorithms for robust interplanetary trajectorydesign,” Theuseofartificialintelligenceforspaceapplications ,editedbyC.Ieracitano,N.Mammone,M.DiClemente, M. Mahmud, R. Furfaro, and F. C. ...

    2023

  53. [61]

    Reaching the limit in autonomous racing: Optimal control versus reinforcement learning,

    Song, Y., Romero, A., Müller, M., Koltun, V., and Scaramuzza, D., “Reaching the limit in autonomous racing: Optimal control versus reinforcement learning,”Science Robotics, Vol. 8, No. 82, 2023, p. eadg1462

    2023

  54. [62]

    Neural Ordinary Differential Equations,

    Chen, R. T. Q., Rubanova, Y., Bettencourt, J., and Duvenaud, D., “Neural Ordinary Differential Equations,”Advances in Neural Information Processing Systems, 2018. 29

    2018

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.