Pith. sign in

REVIEW 5 major objections 5 minor 70 references

Intelligent Control of Spacecraft Reaction Wheel Attitude Using Deep Reinforcement Learning

T0 review · 5 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that TD3-HD, a deep reinforcement learning controller combining TD3, Hindsight Experience Replay, and Dimension-Wise Clipping, keeps a small satellite's attitude error low and angular velocity regulated after one reaction…

desk verdict A well-motivated application paper whose central fault-recovery mechanism is not implementable as stated; the combination is new, but the dynamics, state space, and DWC semantics need serious fixes. read the letter →

arxiv 2507.08366 v1 pith:MQ7BCS63 submitted 2025-07-11 cs.RO cs.AI

Ghaith El-Dalahmeh , Mohammad Reza Jabbarpour , Bao Quoc Vo , Ryszard Kowalczyk This is my paper · ORCID
classification cs.ROcs.AI
keywords DeepReinforcementLearningTwin-DelayedDeterministicPolicyGradientHindsightExperienceReplayAttitudeControlReactionWheelsSpacecraftAutonomyFaultToleranceModifiedRodriguesParameters
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a deep reinforcement learning controller, TD3-HD, can keep a small low Earth orbit satellite's attitude under control when one of its four reaction wheels stops responding, without ground intervention. The authors claim that combining Twin Delayed Deep Deterministic Policy Gradient (TD3) with Hindsight Experience Replay (HER) and Dimension-Wise Clipping (DWC) yields lower attitude error, better angular velocity regulation, and more stable torque redistribution than a PD controller and three standard DRL baselines (PPO, A2C, TD3) in Basilisk simulations with the wheel fault injected at 3000 s. If true, this would give small satellites a practical onboard, fault-tolerant attitude control option that adapts autonomously to actuator failure.

What carries the argument

The mechanism that carries the argument is the TD3-HD actor-critic update with two additions: Hindsight Experience Replay (HER) and Dimension-Wise Clipping (DWC). HER reinterprets unsuccessful episodes by replacing the original goal with a goal the agent actually reached, converting sparse-reward failures into useful training signal. DWC clips the policy gradient independently for each action dimension, so each reaction wheel's torque adjustment is bounded by its own threshold while unaffected wheels keep a clear learning signal. The policy observes only the state $s_t = \{\mathrm{MRP}_{\mathrm{error}}, \omega\}$ and outputs per-wheel torque actions; in fault conditions the action dimension for the unresponsive wheel is clipped and the remaining weighting parameters $\lambda_i$ are renormalized over the functional wheels to redistribute torque, with a backup wheel activated when needed. This combination is what the paper credits for the claimed improvement in fault tolerance and training efficiency.

What would settle it

Run the same trained TD3-HD policy in Basilisk but disable a different reaction wheel, such as RW1 instead of RW0, or shift the fault time; if the state contains no wheel identity or speed, the policy's post-fault torque should not be able to selectively zero the failed wheel, so the claimed redistribution would break and attitude error would grow. A simpler check is to inspect the post-fault torque history for the failed wheel, which should go to zero while healthy wheels compensate; if the failed wheel keeps receiving torque, the mechanism described in Section 4.4 is not what is operating.

Watch

Extended reading notes

Core claim

The central claim is that TD3-HD is a viable fault-tolerant attitude controller for reaction-wheel spacecraft. The paper argues that standard TD3, PPO, and A2C can redistribute torque among the healthy wheels after a fault but converge slowly or oscillate, while TD3-HD's HER component turns sparse-reward failures into learning signal and its DWC component clips each wheel's torque update separately, preventing overcorrection. In the reported 8000-second Basilisk scenario with reaction wheel RW0 disabled at the 3000th second, TD3-HD tracks the desired attitude, expressed in Modified Rodrigues Parameters (MRPs), with consistently low error, damps angular velocity oscillations, and drives the failed wheel's torque to zero while smoothly increasing torque on the remaining wheels. The authors therefore claim TD3-HD outperforms PD control and the three DRL baselines in accuracy, stability, and fault recovery, positioning it as an onboard AI solution for autonomous satellite attitude control.

Load-bearing premise

The load-bearing premise is that the agent can tell which wheel is unresponsive and how to compensate from seeing only the current attitude error and angular velocity, with no fault indicator, wheel speed, or torque feedback in the state.

Editorial extensions

If this is right

  • With the reported performance, a small satellite could continue precise pointing after one reaction wheel fails, avoiding ground intervention for retuning during the mission.
  • HER's ability to learn from sparse rewards means TD3-HD can be trained without dense reward shaping, reducing reward-engineering effort for attitude control tasks.
  • DWC's per-wheel clipping stabilizes torque adjustments so a fault in one wheel does not destabilize policy updates for the other wheels, which is exactly the property needed for actuator fault handling.
  • Standard TD3 already redistributes torque after a wheel fault, but the paper's results indicate that HER removes the slow-convergence bottleneck caused by sparse rewards, making the enhanced variant the practical choice for onboard deployment.
  • The same TD3-HD structure could be applied to other over-actuated spacecraft subsystems, such as thruster or reaction wheel arrays, whenever an actuator can fail partially or completely.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the policy state in Eq. (5) contains only attitude error and angular velocity, the paper never states how the agent identifies which wheel failed. A natural extension the authors do not test is adding wheel-speed or fault-indicator observations to check whether the apparent fault recovery is learned fault identification or merely a fixed response to error growth.
  • The fault is always injected into the same wheel (RW0) at the same time (3000 s), so generalization to other fault wheels, partial degradation, multiple simultaneous faults, or different fault times remains open; testing those variations would show whether TD3-HD's fault tolerance is robust or scenario-specific.
  • The backup wheel's contribution is described but never isolated in the results, so an ablation that toggles HER, DWC, and backup-wheel activation independently would reveal which component actually carries the fault-tolerance improvement.
  • If the partial-observability concern is real, recurrent policies or stacked observations over time could be a more direct fix than gradient clipping, connecting this work to the broader problem of partially observable reinforcement learning.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 5 minor

Summary. The paper proposes TD3-HD, a TD3-based deep reinforcement learning controller augmented with Hindsight Experience Replay (HER) and Dimension-Wise Clipping (DWC), for reaction-wheel attitude control of a small satellite. The authors formulate the problem as an MDP with state {MRP error, angular velocity} and a four-dimensional torque action, and they benchmark TD3-HD against PD, PPO, A2C, and standard TD3 in the Basilisk simulator under a reaction-wheel fault injected at 3000 s. The central claim is that TD3-HD achieves significantly lower attitude error, better angular velocity regulation, and higher stability than the baselines by automatically redistributing torque among the functional wheels. Evaluation is presented as qualitative single-run time histories of attitude error, angular velocity, and wheel torque.

Significance. If the central claim were established, the work would provide a practically relevant demonstration of fault-tolerant, DRL-based attitude control for small LEO satellites, and the use of the high-fidelity Basilisk framework alongside several standard baselines is a reasonable evaluation design. The benchmark is falsifiable, and the paper identifies a genuine operational problem. However, the manuscript as written does not supply a complete, internally consistent algorithm definition, and the stated fault-recovery mechanism is not implementable with the observation vector defined in Eq. (5). These problems affect the central claim rather than the presentation.

major comments (5)
  1. [Section 3.1, Eq. (3)] Eq. (3) declares u_t in R^4 while the left-hand side J dot(omega_t) - S(omega_t) J omega_t is a three-dimensional vector. Since no 3x4 reaction-wheel configuration matrix is defined anywhere in the paper, the mapping from the four wheel torques in Eq. (6) to the three-axis torque entering the dynamics is unspecified. Eq. (7) repeats the problem: T = sum_i lambda_i RW_i is not a vector equation unless the wheel geometry is given. The pyramid arrangement mentioned in Section 5 is never converted into a torque distribution matrix, so the dynamics, the action space, and the claimed torque redistribution cannot be evaluated.
  2. [Section 4.4 and Section 4.3] The fault-recovery mechanism is not implementable with the stated state. Section 4.4 describes clipping the action dimension of an unresponsive wheel and renormalizing lambda_i over the remaining wheels, but the state in Eq. (5), {MRP_error, omega}, contains no fault indicator, no wheel speed, no torque feedback, and no fault-age variable. A feedforward TD3 policy maps the same observation to the same action and therefore cannot know which wheel has failed. In addition, Section 4.3 defines DWC as independent clipping of policy-gradient components during training, while Section 4.4 uses it as a runtime action-masking mechanism; these are two different operations, and the runtime version is asserted without any fault-detection or backup-activation logic.
  3. [Section 4.5, Eq. (10)] The 0.25-degree accuracy-incentive threshold is selected post hoc. The text says the value aligns with industry standards and then states that experimental validation confirmed it, reporting that 0.1 and 0.5 degree alternatives were worse. Because the same simulation and the same reward design are later used as evidence of TD3-HD's superiority, the comparison is circular with respect to this tuned parameter. The authors should either fix the threshold a priori from the cited mission requirements or validate it on a held-out scenario independent of the benchmark comparison.
  4. [Section 6] The abstract's claim of 'significantly lower attitude error, improved angular velocity regulation, and enhanced stability' is not supported by any quantitative statistical comparison. Sections 6.1-6.3 present only single-run plots; no numerical tables of RMS error, settling time, torque variance, or success rate are given, and no multiple-seed trials or confidence intervals are reported. If the small improvements over, for example, standard TD3 are within run-to-run variability, the headline result would not survive. The paper needs seed statistics and a quantitative error metric before the central comparison can be assessed.
  5. [Section 4 and Algorithm 1] The algorithm definition is internally inconsistent and incomplete. Section 4.1 summarizes standard TD3, but Algorithm 1 and the surrounding text introduce importance-sampling weights rho_t, an IS-weighted policy loss J_IS based on KL divergence, Gaussian policy outputs mu_i and sigma_i, and an 'old' network, none of which are defined in the method section. HER is invoked without defining a goal space or goal-conditioned value function, even though HER requires a goal variable to replay hindsight goals. As written, the method cannot be reimplemented independently, and the reported results cannot be attributed to a specific algorithm.
minor comments (5)
  1. [Section 6.2, A2C paragraph] The sentence beginning 'However, a limitation of PPO is...' appears in the A2C performance analysis and should refer to A2C.
  2. [Abstract] The phrase 'the proposed method potential' should be 'the proposed method's potential'.
  3. [Table 6] Several listed parameters, including Trajectory Size N and the four actor sub-networks lambda_i, are not used or explained in the reported experiments.
  4. [Section 3.1] The right-hand side of Eq. (3) should be reconciled with the standard Euler equation; as written, J dot(omega_t) - S(omega_t) J omega_t = u_t has a sign convention that is not stated, and the dimensions of u_t are inconsistent with a three-axis equation.
  5. [Throughout] There are numerous spacing and typographical errors, such as 'R Ws' for reaction wheels, 'DLR' for DRL, and 'challenages' in Section 2.2, that should be corrected in a polished revision.

Circularity Check

1 steps flagged · score 2.0 of 10

Only mild evaluation-loop circularity in the reward threshold; the central benchmark comparison remains empirically grounded.

  1. other [Section 4.5 (Eq. 10) and Sections 6.3/7]
    "The threshold value of 0.25 degrees was selected based on industry standards for high-precision satellite pointing requirements [64]. ... Experimental validation confirmed this threshold as optimal - lower values (e.g., 0.1 degrees) led to excessive control effort and oscillatory behavior without meaningful improvement in steady-state accuracy, while higher thresholds (e.g., 0.5 degrees) resulted in insufficient pointing precision for typical mission requirements."

    Eq. (10) makes a +0.01/-0.01 reward depend on e_current < 0.25, and the 0.25 value is justified in part by "experimental validation" in the same Basilisk setup that later supplies the evidence for TD3-HD's low attitude error. The threshold is therefore selected using the same simulation output that is then cited as evidence of precision, so the reported below-0.25-degree accuracy is partly an evaluation of the tuned reward rather than an independent confirmation of the algorithm. This is a mild evaluation loop rather than a formal derivation, and it does not force the comparative result against PD, PPO, A2C, or TD3.

full rationale

The main claim is an empirical benchmark comparison in Basilisk, not a derivation that reduces to its inputs: the policy update machinery (TD3/HER/DWC) is standard, and the outcome (lower error, better angular velocity regulation, improved fault-tolerance) is reported from simulation rather than constructed from the equations. There are no author self-citations used as load-bearing support. The only self-referential element is the reward threshold in Eq. (10), tuned on the same simulator and then reflected in the measured error, which scores as a mild evaluation loop. The more serious weaknesses—DWC is defined as training-time gradient clipping in Section 4.3 but used as a runtime fault-masking mechanism in Section 4.4, and the state in Eq. (5) contains no fault indicator or wheel-speed information needed for the claimed redistribution—are correctness and implementability concerns, not circularity, and were therefore not added as circular steps.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the simulator's fidelity, the sufficiency of the MRP error and angular velocity state under unobserved faults, standard rigid-body dynamics, and the correctness of third-party RL implementations. The reward thresholds and clipping constants are hand-set or fitted post hoc. No new physical entities are introduced.

free parameters (5)
  • DWC clipping threshold c_i = 0.2
    Table 6 lists a single clipping threshold 0.2 for all dimensions; the algorithm's performance depends on this hand-set value and no sensitivity analysis is given.
  • Reward accuracy incentive threshold = 0.25 degrees
    Equation (10); the paper states experimental validation showed this threshold optimal versus 0.1 and 0.5 degrees, so it is fitted to the same benchmark used for evaluation.
  • Angular velocity penalty threshold = |omega| > 1
    Equation (9); chosen by hand without units or justification; affects training behavior.
  • Reward weights = 1, -10, +-0.01
    Equations (8) to (11); relative weighting of error reduction, velocity penalty, and accuracy incentive is hand-set.
  • TD3-HD training hyperparameters = learning rate 3e-4, batch size 128, hidden units 256, replay 1e6, target update interval 2
    Table 6; standard values claimed to require no extensive tuning, but no ablation or sensitivity check is provided.
assumptions (5)
  • domain assumption Rigid-body rotational dynamics with known inertia J and gyroscopic coupling (Eq. 3) is the correct model.
    The paper provides no validation against hardware or disturbance models; the equation is also dimensionally inconsistent as written.
  • ad hoc to paper The state {MRP_error, omega} is sufficient for optimal torque decisions even when one reaction wheel is unresponsive.
    No fault indicator, wheel speed, or torque feedback is included in the observation, yet Section 4.4 requires the controller to know which wheel is faulty.
  • domain assumption Basilisk simulation accurately represents reaction wheel torque, saturation, and failure behavior.
    Section 5 asserts Basilisk is high fidelity; no calibration or comparison to flight data is given.
  • standard math Standard MRP kinematics (Eq. 1) apply for the rotation ranges considered.
    Standard attitude kinematics; accepted as background, not derived.
  • domain assumption Stable-Baselines3 implementations of TD3, PPO, and A2C are correct and comparable.
    The comparison trusts third-party library implementations; no code or version is pinned.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Intelligent Control of Spacecraft Reaction Wheel Attitude Using Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/MQ7BCS63

@misc{pith2026250708366,
  author       = {Pith},
  title        = {Pith review of: Intelligent Control of Spacecraft Reaction Wheel Attitude Using Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MQ7BCS63}},
  note         = {Machine review of arXiv:2507.08366}
}
read the original abstract

Reliable satellite attitude control is essential for the success of space missions, particularly as satellites increasingly operate autonomously in dynamic and uncertain environments. Reaction wheels (RWs) play a pivotal role in attitude control, and maintaining control resilience during RW faults is critical to preserving mission objectives and system stability. However, traditional Proportional Derivative (PD) controllers and existing deep reinforcement learning (DRL) algorithms such as TD3, PPO, and A2C often fall short in providing the real time adaptability and fault tolerance required for autonomous satellite operations. This study introduces a DRL-based control strategy designed to improve satellite resilience and adaptability under fault conditions. Specifically, the proposed method integrates Twin Delayed Deep Deterministic Policy Gradient (TD3) with Hindsight Experience Replay (HER) and Dimension Wise Clipping (DWC) referred to as TD3-HD to enhance learning in sparse reward environments and maintain satellite stability during RW failures. The proposed approach is benchmarked against PD control and leading DRL algorithms. Experimental results show that TD3-HD achieves significantly lower attitude error, improved angular velocity regulation, and enhanced stability under fault conditions. These findings underscore the proposed method potential as a powerful, fault tolerant, onboard AI solution for autonomous satellite attitude control.

Figures

Figures reproduced from arXiv: 2507.08366 by the authors.

Figure 1
Figure 1. Overview of studied approaches 2.1. Background Various advanced control methods have been applied to spacecraft atti￾tude control, alongside traditional methods like Proportional (P) controllers. These include Sliding Mode Control (SMC) for robustness, adaptive control for handling system variability, fuzzy logic for managing imprecise data, and neural networks for learning complex, and nonlinear dynamics. Each ap￾p… view at source ↗
Figure 2
Figure 2. Interaction of Agent with Environment via Basilisk [PITH_FULL_IMAGE:figures/full_fig_p018_2.png] view at source ↗
Figure 3
Figure 3. Proposed TD3-HD structure. 5. Experimental Setup The Basilisk Astrodynamics Simulation Framework [55] provides a high￾fidelity simulation environment for modeling the kinematics and dynamics of a small satellite in Low Earth Orbit (LEO). Hence, this simulator is used as simulation environment in our experiments The satellite’s orientation is represented by Modified Rodrigues Parameters (MRPs) and controlled using fo… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Simulation configuration used for satellite attitude control. [PITH_FULL_IMAGE:figures/full_fig_p027_4.png]
Figure 5
Figure 5. Figure 5: PD Controller Error Metrics for Attitude Control with RW Fault at 3000 seconds [PITH_FULL_IMAGE:figures/full_fig_p029_5.png]
Figure 6
Figure 6. Figure 6: Torque History of RWs under PD Control with RW0 Fault [PITH_FULL_IMAGE:figures/full_fig_p029_6.png]
Figure 7
Figure 7. Figure 7: PPO Error Metrics for Attitude Control with Fault at 3000 seconds [PITH_FULL_IMAGE:figures/full_fig_p030_7.png]
Figure 8
Figure 8. Figure 8: PPO Torque History for RW0, RW1, RW2, and RW3 [PITH_FULL_IMAGE:figures/full_fig_p031_8.png]
Figure 9
Figure 9. Figure 9: A2C Error Metrics for Attitude Control with Fault at 3000 seconds [PITH_FULL_IMAGE:figures/full_fig_p032_9.png]
Figure 10
Figure 10. Figure 10: A2C Torque History for RW0, RW1, RW2, and RW3 [PITH_FULL_IMAGE:figures/full_fig_p032_10.png]
Figure 11
Figure 11. Figure 11: TD3 Error Metrics for Attitude Control with Fault at 3000 seconds [PITH_FULL_IMAGE:figures/full_fig_p033_11.png]
Figure 12
Figure 12. Figure 12: TD3 Torque History for RW0, RW1, RW2, and RW3 [PITH_FULL_IMAGE:figures/full_fig_p034_12.png]
Figure 13
Figure 13. Figure 13: TD3-HD Error Metrics for Attitude Control with Fault at 3000 seconds [PITH_FULL_IMAGE:figures/full_fig_p035_13.png]
Figure 14
Figure 14. Figure 14: TD3-HD Torque History for RW0, RW1, RW2, and RW3 [PITH_FULL_IMAGE:figures/full_fig_p036_14.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

70 extracted references · 65 canonical work pages

  1. [1]

    J. R. Mansell, Deep learning fault protection applied to spacecraft atti- tude determination and control, Ph.D. thesis, Purdue University (2020)

    work page 2020

  2. [2]

    J. R. Mansell, D. A. Spencer, Deep learning fault diagnosis for spacecraft attitude determination and control, Journal of Aerospace Information Systems 18 (3) (2021) 102–115

    work page 2021

  3. [3]

    Mahfouz, D

    A. Mahfouz, D. Pritykin, J. Biggs, Hybrid attitude control for nano- spacecraft: Reaction wheel failure and singularity handling, Journal of Guidance, Control, and Dynamics 44 (3) (2021) 548–558

    work page 2021

  4. [4]

    Miralles, K

    P. Miralles, K. Thangavel, A. F. Scannapieco, N. Jagadam, P. Baranwal, B. Faldu, R. Abhang, S. Bhatia, S. Bonnart, I. Bhatnagar, et al., A critical review on the state-of-the-art and future prospects of machine learning for earth observation operations, Advances in Space Research 71 (12) (2023) 4959–4986

    work page 2023

  5. [5]

    Fujimoto, H

    S. Fujimoto, H. Hoof, D. Meger, Addressing function approximation error in actor-critic methods, in: International conference on machine learning, PMLR, 2018, pp. 1587–1596

    work page 2018

  6. [6]

    Andrychowicz, F

    M. Andrychowicz, F. Wolski, A. Ray, J. Schneider, R. Fong, P. Welin- der, B. McGrew, J. Tobin, O. Pieter Abbeel, W. Zaremba, Hindsight experience replay, Advances in neural information processing systems 30 (2017)

    work page 2017

  7. [7]

    H. Wu, H. Ye, W. Xue, X. Yang, Improved reinforcement learning using stability augmentation with application to quadrotor attitude control, IEEE Access 10 (2022) 67590–67604

    work page 2022

  8. [8]

    Henna, H

    H. Henna, H. Toubakh, M. R. Kafi, M. Sayed-Mouchaweh, Towards fault-tolerant strategy in satellite attitude control systems: A review, in: Annual Conference of the PHM Society, Vol. 12, 2020, pp. 14–14

    work page 2020

Show all 70 references
  1. [9]

    M. F. Mehrjardi, H. Sanusi, M. A. M. Ali, M. A. Taher, Pd controller for three-axis satellite attitude control using discrete kalman filter, in: 2014 International Conference on Computer, Communications, and Control Technology (I4CT), IEEE, 2014, pp. 83–85. 39

    2014

  2. [10]

    K. M. Mohan, U. Anitha, K. Anbumani, Cubesat attitude control by implementation of pid controller using python, in: 2023 12th Interna- tional Conference on Advanced Computing (ICoAC), IEEE, 2023, pp. 1–5

    2023

  3. [11]

    M. L. Orozco, B. S. Giraldo, Attitude determination and control in small satellites: A review, IEEE Journal on Miniaturization for Air and Space Systems (2024)

    2024

  4. [12]

    T. Wu, C. Zhou, Z. Yan, H. Peng, L. Wu, Application of pid opti- mization control strategy based on particle swarm optimization (pso) for battery charging system, International Journal of Low-Carbon Tech- nologies 15 (4) (2020) 528–535

    2020

  5. [13]

    A. K. Parsai, J. S. Qureishi, G. Raju, K. Ratnakara, Model based pid tuning of antenna control system for tracking of spacecraft, in: 2019 3rd International conference on Electronics, Communication and Aerospace Technology (ICECA), IEEE, 2019, pp. 906–913

    2019

  6. [14]

    Z. Zhu, Y. Xia, M. Fu, Adaptive sliding mode control for attitude sta- bilization with actuator saturation, IEEE Transactions on Industrial Electronics 58 (10) (2011) 4898–4907

    2011

  7. [15]

    B. Wang, S. Li, Q. Zhang, M. Xin, Combined fuzzy sliding-mode atti- tude stabilization and energy storage for small satellite, IEEE Transac- tions on Aerospace and Electronic Systems (2023)

    2023

  8. [16]

    L. Cao, X. Chen, A. K. Misra, Minimum sliding mode error feedback control for fault tolerant reconfigurable satellite formations with j2 per- turbations, Acta Astronautica 96 (2014) 201–216

    2014

  9. [17]

    Modirrousta, M

    A. Modirrousta, M. Khodabandeh, Adaptive non-singular terminal slid- ing mode controller: new design for full control of the quadrotor with external disturbances, Transactions of the Institute of Measurement and Control 39 (3) (2017) 371–383

    2017

  10. [18]

    W. Xin, Z. Shasha, Z. Xingwang, Adaptive fuzzy sliding mode controller for attitude coordinated control in spacecraft formation, GSTF Journal on Aviation Technology (JAT) 1 (2015) 1–7. 40

    2015

  11. [19]

    S. Jia, J. Shan, Continuous integral sliding mode control for space ma- nipulator with actuator uncertainties, Aerospace Science and Technol- ogy 106 (2020) 106192

    2020

  12. [20]

    H. Wang, L. Chen, Z. Jin, J. L. Crassidis, Adaptive momentum dis- tribution jitter control for microsatellite, Journal of Guidance, Control, and Dynamics 42 (3) (2019) 632–641

    2019

  13. [21]

    R. A. Freeman, P. Kokotovi´ c, Backstepping design of robust controllers for a class of nonlinear systems, in: Nonlinear Control Systems Design 1992, Elsevier, 1993, pp. 431–436

    1992

  14. [22]

    Babaei Faramarz, A

    S. Babaei Faramarz, A. Akbarzadeh Kalat, An output feedback back- stepping attitude control for rigid satellite, Transactions of the Institute of Measurement and Control 45 (11) (2023) 2182–2191

    2023

  15. [23]

    K. Yan, Q. Wu, C. Yang, M. Chen, Backstepping-based adaptive fault- tolerant control design for satellite attitude system, in: 2020 Interna- tional Conference on Unmanned Aircraft Systems (ICUAS), IEEE, 2020, pp. 176–181

    2020

  16. [24]

    Boussadia, A

    H. Boussadia, A. S. Mohammed, N. Boughanmi, A. Bellar, Adaptive backstepping control for microsatellite under inertia uncertainties, in: 2017 8th International Conference on Recent Advances in Space Tech- nologies (RAST), IEEE, 2017, pp. 67–72

    2017

  17. [25]

    Feedback and optimal sensitivity: Model reference transformations, mul- tiplicative seminorms, and approximate inverses, IEEE Transactions on automatic control 26 (2) (1981) 301–320

    1981

  18. [26]

    Z. Wang, Y. Li, Rigid spacecraft nonlinear robust h∞ attitude controller design under actuator misalignments, Nonlinear Dynamics 111 (16) (2023) 15037–15054

    2023

  19. [27]

    S. M. Esmaeilzadeh, M. S. Zeyghami, Nonlinear finite time attitude con- trol of flexible spacecraft based on a novel output redefinition method, Chinese Journal of Aeronautics 36 (11) (2023) 373–385

    2023

  20. [28]

    T. H. Davenport, R. Ronanki, et al., Artificial intelligence for the real world, Harvard business review 96 (1) (2018) 108–116. 41

    2018

  21. [29]

    P. J. Werbos, Backpropagation through time: what it does and how to do it, Proceedings of the IEEE 78 (10) (1990) 1550–1560

    1990

  22. [30]

    Carrara, A

    A. Carrara, A. R. Neto, Satellite attitude acquisition using a neural network controller, Advances in space dynamics; Advances in Space Dy- namics (2000) 272–282

    2000

  23. [31]

    Bellman, The theory of dynamic programming, Bulletin of the Amer- ican Mathematical Society 60 (6) (1954) 503–515

    R. Bellman, The theory of dynamic programming, Bulletin of the Amer- ican Mathematical Society 60 (6) (1954) 503–515

    1954

  24. [32]

    V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al., Human-level control through deep reinforcement learning, nature 518 (7540) (2015) 529–533

    2015

  25. [33]

    Lillicrap, Continuous control with deep reinforcement learning, arXiv preprint arXiv:1509.02971 (2015)

    T. Lillicrap, Continuous control with deep reinforcement learning, arXiv preprint arXiv:1509.02971 (2015)

    2015 arXiv

  26. [34]

    Haarnoja, A

    T. Haarnoja, A. Zhou, P. Abbeel, S. Levine, Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic ac- tor, in: International conference on machine learning, PMLR, 2018, pp. 1861–1870

    2018

  27. [35]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, O. Klimov, Proximal policy optimization algorithms, arXiv preprint arXiv:1707.06347 (2017)

    2017 arXiv

  28. [36]

    Schulman, Trust region policy optimization, arXiv preprint arXiv:1502.05477 (2015)

    J. Schulman, Trust region policy optimization, arXiv preprint arXiv:1502.05477 (2015)

    2015 arXiv

  29. [37]

    Y. Chai, J. Luo, N. Han, Spacecraft attitude analytical predictive control based on sequential action control, in: 2018 IEEE CSAA Guidance, Navigation and Control Conference (CGNCC), IEEE, 2018, pp. 1–7

    2018

  30. [38]

    Y. Yang, K. Caluwaerts, A. Iscen, T. Zhang, J. Tan, V. Sindhwani, Data efficient reinforcement learning for legged robots, in: Conference on Robot Learning, PMLR, 2020, pp. 1–10

    2020

  31. [39]

    X. He, C. Lv, Robotic control in adversarial and sparse reward envi- ronments: A robust goal-conditioned reinforcement learning approach, IEEE Transactions on Artificial Intelligence 5 (1) (2023) 244–253. 42

    2023

  32. [40]

    Hovell, S

    K. Hovell, S. Ulrich, On deep reinforcement learning for spacecraft guid- ance, in: AIAA Scitech 2020 forum, 2020, p. 1600

    2020

  33. [41]

    Y. Liu, G. Ma, Y. Lyu, P. Wang, Neural network-based reinforcement learning control for combined spacecraft attitude tracking maneuvers, Neurocomputing 484 (2022) 67–78

    2022

  34. [42]

    Y. Meng, C. Liu, Y. Liu, L. Tan, Adaptive fault-tolerant control for spacecraft: A dynamic stackelberg game approach with a2c reinforce- ment learning, Available at SSRN 4683974

  35. [43]

    Retagne, J

    W. Retagne, J. Dauer, G. Waxenegger-Wilfing, Adaptive satellite atti- tude control for varying masses using deep reinforcement learning, Fron- tiers in Robotics and AI 11 (2024) 1402846

    2024

  36. [44]

    Henna, Attitude fault-tolerant control applied to microsatellite, Ph.D

    H. Henna, Attitude fault-tolerant control applied to microsatellite, Ph.D. thesis, UNIVERSITY OF KASDI MERBAH OUARGLA (2024)

    2024

  37. [45]

    Gaudet, R

    B. Gaudet, R. Linares, R. Furfaro, Deep reinforcement learning for six degree-of-freedom planetary landing, Advances in Space Research 65 (7) (2020) 1723–1741

    2020

  38. [46]

    Zhang, X

    Z. Zhang, X. Li, J. An, W. Man, G. Zhang, Model-free attitude control of spacecraft based on pid-guide td3 algorithm, International Journal of Aerospace Engineering 2020 (1) (2020) 8874619

    2020

  39. [47]

    J.-A. R. Sarmiento, V. H. Tan, M. C. R. Talampas, P. C. Naval Jr, Sample efficient deep reinforcement learning for diwata microsatellite reaction wheel attitude control, Aerospace Systems 6 (1) (2023) 61–69

    2023

  40. [48]

    N. A. Chaturvedi, A. K. Sanyal, N. H. McClamroch, Rigid-body attitude control, IEEE control systems magazine 31 (3) (2011) 30–51

    2011

  41. [49]

    Bandyopadhyay, S.-J

    S. Bandyopadhyay, S.-J. Chung, F. Hadaegh, Attitude control and sta- bilization of spacecraft with a captured asteroid, in: AIAA Guidance, Navigation, and Control Conference, 2015, p. 0596

    2015

  42. [50]

    Calaon, H

    R. Calaon, H. Schaub, Constrained attitude maneuvering via modified- rodrigues-parameter-based motion planning algorithms, Journal of Spacecraft and Rockets 59 (4) (2022) 1342–1356. 43

    2022

  43. [51]

    F. L. Markley, J. L. Crassidis, Attitude kinematics and dynamics, in: Fundamentals of Spacecraft Attitude Determination and Control, Springer, 2014, pp. 67–122

    2014

  44. [52]

    Y. Su, C. Zheng, Globally asymptotic stabilization of spacecraft with simple saturated proportional-derivative control, Journal of Guidance, Control, and Dynamics 34 (6) (2011) 1932–1936

    2011

  45. [53]

    P. C. Hughes, Spacecraft attitude dynamics, Courier Corporation, 2012

    2012

  46. [54]

    Elkins, R

    J. Elkins, R. Sood, C. Rumpf, Adaptive continuous control of spacecraft attitude using deep reinforcement learning, in: Proceedings of, 2020, pp. 420–475

    2020

  47. [55]

    P. W. Kenneally, S. Piggott, H. Schaub, Basilisk: A flexible, scalable and modular astrodynamics simulation framework, Journal of aerospace information systems 17 (9) (2020) 496–507

    2020

  48. [56]

    M. Fang, C. Zhou, B. Shi, B. Gong, J. Xu, T. Zhang, Dher: Hindsight experience replay for dynamic goals, in: International Conference on Learning Representations, 2018

    2018

  49. [57]

    R. Liu, F. Nageotte, P. Zanne, M. de Mathelin, B. Dresp-Langley, Deep reinforcement learning for the control of robotic manipulation: a fo- cussed mini-review, Robotics 10 (1) (2021) 22

    2021

  50. [58]

    Creus-Castanyer, Intrinsic exploration for reinforcement learning be- yond rewards (2024)

    R. Creus-Castanyer, Intrinsic exploration for reinforcement learning be- yond rewards (2024)

    2024

  51. [59]

    Narvekar, B

    S. Narvekar, B. Peng, M. Leonetti, J. Sinapov, M. E. Taylor, P. Stone, Curriculum learning for reinforcement learning domains: A framework and survey, Journal of Machine Learning Research 21 (181) (2020) 1–50

    2020

  52. [60]

    Zhang, Gradient-norm constrained algorithm on offline and online learning, Available at SSRN 4663051

    H. Zhang, Gradient-norm constrained algorithm on offline and online learning, Available at SSRN 4663051

  53. [61]

    Zhang, J

    Y. Zhang, J. Sun, G. Wang, J. Chen, Addressing high-dimensional con- tinuous action space via decomposed discrete policy-critic (2022)

    2022

  54. [62]

    S. Han, Y. Sung, Dimension-wise importance sampling weight clipping for sample-efficient reinforcement learning, in: International Conference on Machine Learning, PMLR, 2019, pp. 2586–2595. 44

    2019

  55. [63]

    Arulkumaran, M

    K. Arulkumaran, M. P. Deisenroth, M. Brundage, A. A. Bharath, Deep reinforcement learning: A brief survey, IEEE Signal Processing Maga- zine 34 (6) (2017) 26–38

    2017

  56. [64]

    Sawada, T

    H. Sawada, T. Hashimoto, K. Ninomiya, High-stability attitude con- trol of satellites by magnetic bearing wheels, Transactions of the Japan Society for Aeronautical and Space Sciences 44 (145) (2001) 133–141

    2001

  57. [65]

    J. R. Forbes, Fundamentals of spacecraft attitude determination and control [bookshelf], IEEE Control Systems Magazine 35 (4) (2015) 56– 58

    2015

  58. [66]

    A. A. Al-Atawi, Genetically optimized td3 algorithm for efficient access control in the internet of vehicles, Wireless Networks (2024) 1–21

    2024

  59. [67]

    W. Xiao, L. Yuan, T. Ran, L. He, J. Zhang, J. Cui, Multimodal fusion for autonomous navigation via deep reinforcement learning with sparse rewards and hindsight experience replay, Displays 78 (2023) 102440

    2023

  60. [68]

    V. Tan, J. L. Labrador, M. C. Talampas, Mata-rl: continuous reaction wheel attitude control using the mata simulation software and reinforce- ment learning (2021)

    2021

  61. [69]

    Raffin, A

    A. Raffin, A. Hill, A. Gleave, A. Kanervisto, M. Ernestus, N. Dor- mann, Stable-baselines3: Reliable reinforcement learning implementa- tions, Journal of Machine Learning Research 22 (268) (2021) 1–8

    2021

  62. [70]

    Brockman, V

    G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, W. Zaremba, Openai gym (2016). arXiv:arXiv:1606.01540. 45

    2016 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.