Pith. sign in

REVIEW 4 major objections 6 minor 49 references

A Domain-Knowledge-Aided Deep Reinforcement Learning Approach for Flight Control Design

T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A deep reinforcement learning agent that learns the gains of a fixed three-loop autopilot can match or beat classical gain scheduling and tolerate ±40% aerodynamic uncertainty.

desk verdict A useful feasibility study on learning missile autopilot gains with DDPG, but the generalization claim outruns the evidence. read the letter →

arxiv 1908.06884 v2 pith:MN4L4ALW submitted 2019-08-19 cs.AI cs.LGcs.SYeess.SY

Hyo-Sang Shin , Shaoming He , Antonios Tsourdos This is my paper · ORCID
classification cs.AIcs.LGcs.SYeess.SY
keywords flightcontroldeepreinforcementlearningdeterministicpolicygradientthree-loopautopilotgainschedulingrewardshapingmodeluncertainty
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes that a deep reinforcement learning agent can design a flight-control autopilot by learning the gains of a fixed three-loop autopilot structure, rather than learning control commands from scratch. The agent maps directly measurable flight conditions—angle of attack, Mach number, and height—to four autopilot gains, and the reward function is built around a shaped reference command that encodes desired transient behavior. Numerical simulations in the paper show that the resulting deep deterministic policy gradient (DDPG) autopilot tracks lateral-acceleration commands across a flight envelope, keeps tracking quality when aerodynamic coefficients are perturbed by up to ±40%, and meets conventional gain- and phase-margin criteria. If these findings hold, a data-driven gain schedule could replace manually tuned look-up tables while preserving the simple structure of classical autopilots.

What carries the argument

The load-bearing mechanism is the classical three-loop autopilot structure: an inner proportional–integral loop on pitch rate plus an outer proportional loop on lateral acceleration. DDPG learns the four gains of this structure as a static nonlinear function of the observation $(\alpha, M, h)$, with normalized inputs and outputs. The reward function is another use of domain knowledge: instead of penalizing raw tracking error, the agent tracks a shaped reference command generated by a non-minimum-phase reference system with transfer function $(-0.0363s+1)/(0.009s^2+0.33s+1)$, with penalties on normalized tracking error and fin deflection rate. This shaping aligns the reinforcement learning objective with control-design objectives such as rise time, overshoot, and damping.

What would settle it

Simulate the trained autopilot at one fixed flight condition on step commands of 20, 50, 100, 150, and 200 m/$s^{2}$, with the same initial conditions used in training. If tracking accuracy or stability degrades sharply away from 100 m/$s^{2}$, the generalization claim is refuted; if performance holds across the full range, the claim is supported.

Watch

Extended reading notes

Core claim

The paper's central claim is that a fixed-structure three-loop lateral-acceleration autopilot can be tuned entirely by DDPG, producing a nonlinear gain schedule $K_{DC}$, $K_A$, $K_I$, $K_g$ as a function of $(\alpha, M, h)$ that matches or improves on classical gain scheduling. The authors show that the trained agent tracks a 100 m/s$^2$ step command from randomly initialized states across the flight envelope, generalizes to reference signals that differ from the training command, and maintains steady-state tracking accuracy when aerodynamic coefficients are randomly perturbed by ±40% without retraining. They also report that numerical gain-margin and phase-margin tests at several heights satisfy the typical design criteria of more than 6 dB and more than 45 degrees.

Load-bearing premise

The learned gains depend only on angle of attack, Mach number, and height, not on the commanded acceleration or on the current tracking error, so the claim that one gain schedule trained on a single step command generalizes to all commands in the envelope rests on the assumption that the optimal gains are command-independent.

Editorial extensions

If this is right

  • If the central claim holds, autopilot gains can be produced from simulation data without linearizing the airframe at multiple trim points or building an interpolation look-up table.
  • The same approach could be applied to other fixed-structure controllers and other vehicles by swapping the structure and the reward, making DRL a general 'gain tuner' rather than a black-box controller.
  • Because the learned policy is a mapping from measurable flight conditions to gains, the resulting controller can still be implemented and analyzed like a conventional scheduled autopilot, easing practical adoption.
  • The ±40% uncertainty results suggest that a DDPG-trained fixed-structure autopilot can tolerate significant aerodynamic model error, pointing toward data-driven designs where the plant model is used only for training.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The training uses a single 100 m/s$^2$ step command, so the claim that the agent generalizes to other commands deserves a direct test across command magnitudes and signs; the observation vector omits the command value, and the argument for command-independence rests on the integral action in the inner loop.
  • A useful extension would be to include the commanded acceleration or the tracking error in the observation; if the optimal gains turn out to depend on command size, the current observation set would be insufficient and the reported generalization would not cover large parts of the envelope.
  • The phase-margin estimate is computed from the time response of an injected time delay rather than from a frequency-response measurement; checking the margins with a frequency-domain analysis of the linearized loop at the same set points would verify that the reported 45-degree figure matches the standard definition.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a deep reinforcement learning (DRL) approach to lateral acceleration autopilot design for a tail-controlled airframe. Instead of learning the control input directly, the authors fix the classical three-loop autopilot structure and use DDPG to learn a deterministic mapping from the flight condition (angle of attack, Mach number, height) to the four autopilot gains KDC, KA, KI, and Kg. A domain-knowledge-based reward is constructed by shaping the reference command through a desired reference model and penalizing tracking error and fin deflection rate. The agent is trained on random initial conditions in the flight envelope under a fixed 100 m/s^2 step command. Test results compare the learned autopilot with a classical gain-scheduling baseline, examine robustness under +/- 40% aerodynamic coefficient uncertainty, and numerically evaluate gain and phase margins. The paper claims improved learning efficiency and generalizability relative to learning from scratch and several practical advantages over gain scheduling.

Significance. If the claims are substantiated, the paper makes a useful contribution to the growing literature on DRL in flight control by showing that a fixed-structure autopilot with learned gains can be trained across a flight envelope and can provide tracking performance comparable to a gain-scheduled design. The idea of injecting domain knowledge through the controller structure and through reference-command shaping is sensible and practically motivated, and the empirical demonstration of convergence with random initial conditions is a valuable positive result. The paper does not provide code, formal guarantees, or multi-seed statistics, so the significance rests almost entirely on the presented simulations; the generalization and robustness claims therefore need to be backed by more rigorous evidence before the results can be considered established.

major comments (4)
  1. [Section VI-B and Section IV-B] The claim that the domain-knowledge agent 'can track reference signals which differ from those in the training process' is not supported by any experiment reported in the paper. The training scenario in Section V-1 uses the same 100 m/s^2 step command for all episodes, and the test results in Figs. 7-10 appear to use the same command magnitude; no results for a different step magnitude, a ramp, a sine, or any time-varying command are given. Since the agent observation in Eq. (24) is only (alpha, M, h) and does not include the commanded acceleration or any tracking-error state, the learned gains are a static function of the flight condition alone, so it is a load-bearing premise that a gain schedule trained on one command generalizes to all commands in the envelope. The integral action in the three-loop structure can only guarantee zero steady-state error for constant commands; transient metrics such as overshoot, settling time, and actuator activity generally depend on command magnitude and shape. Please either provide quantitative experiments with varied reference signals or explicitly restrict the claimed contribution to tracking of step commands within the training envelope.
  2. [Section VI-A and VI-B] The training and testing are based on a single DDPG run. Learning curves in Figs. 5 and 6 show a single trace (smoothed over 30 episodes), and the test agent is selected as the one with the largest episode reward during that one run. DDPG is initialized stochastically and exhibits high variance across runs and across random seeds; without multiple seeds and reported mean/variance or a reproducibility statement, the convergence behavior, the reward values, and the test comparisons cannot be assessed as reliable. This is particularly important because the numerical comparison with gain scheduling in Section VI-B rests on the performance of one selected policy.
  3. [Section VI-B] The gain-scheduling baseline is underdescribed. The text mentions that the gains at a set point are designed to meet typical autopilot design criteria, but it does not specify the set-point grid, the linearization points, the interpolation algorithm, or the exact design method. In the same paragraph the authors state that the set points 'are not optimally selected,' which makes it difficult to interpret the comparison: the proposed approach is claimed to require less actuator activity and to produce less oscillatory responses than gain scheduling, but the baseline may be at a disadvantage by construction. Please provide a fully specified baseline that can be reproduced, or frame the comparison as illustrative rather than as evidence of a systematic advantage.
  4. [Section VI-C] The robustness study does not report the number of Monte Carlo trials or the uncertainty distribution. The text says the aerodynamic coefficients have 'random -40% to +40% uncertainty,' and Fig. 11 shows a single time response. A single realization is insufficient to support the claim that the autopilot is 'robust against model uncertainty' across the envelope. Please run a Monte Carlo study with many realizations and report e.g. the envelope of responses, worst-case overshoot, and the distribution of steady-state errors, with the uncertainty explicitly described.
minor comments (6)
  1. [Section V-2] There is a typographical error in the text: 'unitless observations hat belong' should read 'unitless observations that belong.'
  2. [Fig. 5] The caption refers to 'the first row' of Fig. 5, but the figure appears to contain a single row of three plots; please correct the reference or the figure layout.
  3. [Section II-B and Algorithm 1] The pseudocode in Algorithm 1 does not close the inner for-loop (line 5) before the outer for-loop (line 4); please add an 'end for' statement for correctness.
  4. [Section VI-D, Eq. (35)] The phase margin expression PM = 360 f Δt deserves a brief derivation or a caveat. For a linear system, the phase lag introduced by a delay at frequency f is 360 f Δt, but in the nonlinear DRL setting the frequency read from the oscillation response is only an approximate estimate of the gain crossover frequency; please state this limitation explicitly.
  5. [Section III, Eq. (17)] The units of the aerodynamic polynomial coefficients in Table II and of the normalization constants in Table VI (e.g., ˙δmax = 1.5) are not stated; please add units to make the model and the reward scaling reproducible.
  6. [References] Reference [47] (Duan et al.) is cited to support the applicability of DDPG to partially observable MDPs, but that paper is a benchmark of continuous-control algorithms in fully observable environments; please provide a more direct reference for memoryless policies in POMDPs or justify the statement in the text.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: learned gains are validated on test scenarios and held-out uncertainties, and all design choices are stated inputs rather than fitted outputs.

full rationale

I walked the paper's derivation chain and found no step in which an output is equivalent to an input by construction. The autopilot gains KDC, KA, KI, Kg are learned by DDPG from a reward function that penalizes tracking error against a shaped reference command and fin deflection rate (Eqs. 25-26); the reward weights, normalization constants, reference-model coefficients, and network hyperparameters are hand-chosen design inputs, not quantities fitted to the subsequently reported tracking, robustness, or stability results. The trained agent is then evaluated on random scenarios and compared with classical gain scheduling (Section VI-B), under aerodynamic uncertainties deliberately excluded from training (Section VI-C), and through numerically computed gain and phase margins (Section VI-D). None of these evaluations feeds back into the training objective or renames a fitted parameter as a prediction. The three-loop structure and reference model are adopted from classical control knowledge and stated as assumptions, not derived from the paper's own results. Although the observation vector o_t = (alpha, M, h) excludes the commanded acceleration and tracking error, which raises a legitimate generalization concern, that is a correctness and validation issue rather than a circularity issue; it does not make any claimed prediction equivalent to an input. There is also no load-bearing self-citation chain or uniqueness theorem imported from the authors' prior work. Accordingly, the appropriate finding is no significant circularity.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

No new physical entities are introduced; the shaped reference command is a mathematical signal used in the reward, not a postulated physical quantity. The free parameters are hand-chosen design inputs that shape the learning objective and action range, and the axioms are the modeling and algorithmic assumptions the empirical claims rest on.

free parameters (6)
  • Reward weight ka = 1
    Weight on squared tracking error to the shaped command in Eq. (26); chosen by hand, balances tracking accuracy against fin rate.
  • Reward weight k_delta = 0.1
    Weight on squared fin deflection rate in Eq. (26); hand-tuned to limit actuator usage.
  • Reference model numerator coefficient = -0.0363
    Non-minimum phase zero of the shaping filter in Eq. (25); chosen to reflect the tail-controlled airframe's NMP property, no tuning criterion given.
  • Reference model denominator coefficients = 0.009 and 0.33
    Set natural frequency and damping of the shaped reference command in Eq. (25); encode desired transient performance, hand-picked.
  • Action normalization constants = [3, 0.05, 100, 2]
    Table VI scaling for KDC, KA, KI, Kg at the actor output; chosen by hand to keep normalized actions in roughly (-1,1).
  • Exploration noise initial variance = 0.1
    Table VII; Ornstein-Uhlenbeck noise variance at start, chosen by trial and error.
assumptions (4)
  • domain assumption The longitudinal airframe model in Eqs. (16)-(20) with aerodynamic polynomials from Table II accurately represents the flight vehicle's behavior.
    All training and testing are against this model; the paper does not validate against flight data or a higher-fidelity model.
  • domain assumption The classical three-loop autopilot topology is a suitable controller structure for this airframe across the flight envelope.
    The paper fixes this structure based on prior missile autopilot literature [1], [14], [45], [46]; it does not learn the topology.
  • domain assumption DDPG with the reported hyperparameters converges to a policy that generalizes over the flight envelope.
    Convergence is shown for one training run with random initial states; no multiple-seed statistics are reported, and DRL training is known to be seed-sensitive.
  • ad hoc to paper The shaped reference system in Eq. (25) represents desirable closed-loop behavior for a tail-controlled airframe.
    The coefficients are introduced specifically to define the reward; no independent justification or comparison with alternative shaping filters is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Domain-Knowledge-Aided Deep Reinforcement Learning Approach for Flight Control Design." pith.science (2026). https://pith.science/paper/MN4L4ALW

@misc{pith2026190806884,
  author       = {Pith},
  title        = {Pith review of: A Domain-Knowledge-Aided Deep Reinforcement Learning Approach for Flight Control Design},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MN4L4ALW}},
  note         = {Machine review of arXiv:1908.06884}
}
read the original abstract

This paper aims to examine the potential of using the emerging deep reinforcement learning techniques in flight control. Instead of learning from scratch, we suggest to leverage domain knowledge available in learning to improve learning efficiency and generalisability. More specifically, the proposed approach fixes the autopilot structure as typical three-loop autopilot and deep reinforcement learning is utilised to learn the autopilot gains. To solve the flight control problem, we then formulate a Markovian decision process with a proper reward function that enable the application of reinforcement learning theory. Another type of domain knowledge is exploited for defining the reward function, by shaping reference inputs in consideration of important control objectives and using the shaped reference inputs in the reward function. The state-of-the-art deep deterministic policy gradient algorithm is utilised to learn an action policy that maps the observed states to the autopilot gains. Extensive empirical numerical simulations are performed to validate the proposed computational control algorithm.

Figures

Figures reproduced from arXiv: 1908.06884 by the authors.

Figure 1
Figure 1. Basic concept of DDPG. where Qw (st, at) stands for the action-value function, which is parameterised by w. The parameter µ is then updated by moving the policy in the direction of the gradient of Qw in a recursive way as µt+1 = µt + αµ∇µA µ (st) ∇atQ w (st, at) (6) where αµ refers to the learning rate of the actor network. Similar to Q-learning, DDPG also utilises the temporal￾difference (TD) error δt in approximat… view at source ↗
Figure 2
Figure 2. The longitudinal dynamics model and parameter definitions. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Three-loop autopilot structure. scheduling the gains with some external signals, e.g., angle￾of-attack, Mach number, height in linear control. Due to this fact, implementing classical three-loop autopilot requires a look-up table and a proper scheduling algorithm. This fact inevitably increases the complexity of the controller and results in some approximation errors during the scheduling process. For modern air veh… view at source ↗
Figures from the paper (11 more)
Figure 4
Figure 4. Figure 4: Information flow of the proposed RL framework. [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Comparisons of learning curves. 0 200 400 600 800 1000 -2000 -1500 -1000 -500 0 [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Learning process comparison with respect to normalisation. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Comparison results of acceleration response. [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Comparison results of angle-of-attack. 0 1 2 3 4 5 6 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 (a) Gain-scheduling 0 1 2 3 4 5 6 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 (b) DDPG without shaped reference command 0 1 2 3 4 5 6 2.6 2.7 2.8 2.9 3 3.1 3.2 3.3 (c) DDPG with shaped ref…
Figure 9
Figure 9. Figure 9: Comparison results of Mach number. The aerodynamic coefficients, detailed in Table II, are assumed to have random −40% to +40% uncertainty. To better show the robustness of the proposed control algorithm, these model uncertainties are not included in the training scena…
Figure 10
Figure 10. Figure 10: Comparison results of fin deflection angle. [PITH_FULL_IMAGE:figures/full_fig_p011_10.png]
Figure 11
Figure 11. Figure 11: Simulation results of the proposed autopilot with model uncertainties. [PITH_FULL_IMAGE:figures/full_fig_p011_11.png]
Figure 12
Figure 12. Figure 12: Method to determine gain margin. the control loop to guarantee stability. With these concepts in mind, the gain margin can by numerically computed as az,c e −∆ts az Time delay Sensor Autopilot Actuator Airframe [PITH_FULL_IMAGE:figures/full_fig_p011_12.png]
Figure 14
Figure 14. Figure 14: Relative stability analysis with gain margin. [PITH_FULL_IMAGE:figures/full_fig_p013_14.png]
Figure 15
Figure 15. Figure 15: Relative stability analysis with phase margin. [PITH_FULL_IMAGE:figures/full_fig_p013_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 45 canonical work pages

  1. [1]

    Zarchan, Tactical and strategic missile guidance

    P. Zarchan, Tactical and strategic missile guidance. American Institute of Aeronautics and Astronautics, 2012

    work page 2012

  2. [2]

    State-space interpolation for a gain-scheduled autopilot,

    D. J. Stilwell, “State-space interpolation for a gain-scheduled autopilot,” Journal of Guidance, Control, and Dynamics , vol. 24, no. 3, pp. 460– 465, 2001

    work page 2001

  3. [3]

    Interpolation of observer state feed- back controllers for gain scheduling,

    D. J. Stilwell and W. J. Rugh, “Interpolation of observer state feed- back controllers for gain scheduling,” IEEE Transactions on Automatic Control, vol. 44, no. 6, pp. 1225–1229, 1999

    work page 1999

  4. [4]

    Missile autopilot design: gain-scheduling and the gap metric,

    S. Theodoulis and G. Duc, “Missile autopilot design: gain-scheduling and the gap metric,” Journal of Guidance, Control, and Dynamics , vol. 32, no. 3, pp. 986–996, 2009

    work page 2009

  5. [5]

    Gain-scheduling control design in the presence of hidden coupling terms,

    H. Lhachemi, D. Saussi ´e, and G. Zhu, “Gain-scheduling control design in the presence of hidden coupling terms,”Journal of Guidance, Control, and Dynamics, pp. 1872–1880, 2016

    work page 2016

  6. [6]

    A sliding mode missile pitch autopilot synthesis for high angle of attack maneuvering,

    A. Thukral and M. Innocenti, “A sliding mode missile pitch autopilot synthesis for high angle of attack maneuvering,” IEEE Transactions on Control Systems Technology, vol. 6, no. 3, pp. 359–371, 1998

    work page 1998

  7. [7]

    Robust missile autopilot design via high-order sliding mode control,

    I. Shkolnikov, Y . Shtessel, D. Lianos, and A. Thies, “Robust missile autopilot design via high-order sliding mode control,” inAIAA Guidance, navigation, and control Conference and Exhibit , 2000

    work page 2000

  8. [8]

    Nonlinear autopilot design for an asym- metric missile using robust backstepping control,

    G. Mattei and S. Monaco, “Nonlinear autopilot design for an asym- metric missile using robust backstepping control,” Journal of Guidance, Control, and Dynamics , vol. 37, no. 5, pp. 1462–1476, 2014

    work page 2014

Show all 49 references
  1. [9]

    Adaptive autopilot design for guided munitions,

    A. J. Calise, M. Sharma, and J. E. Corban, “Adaptive autopilot design for guided munitions,” Journal of Guidance, Control, and Dynamics , vol. 23, no. 5, pp. 837–843, 2000

    2000

  2. [10]

    L1 adaptive controller for a missile longitudinal autopilot design,

    J. Wang, C. Cao, N. Hovakimyan, R. Hindman, and D. B. Ridgely, “ L1 adaptive controller for a missile longitudinal autopilot design,” in AIAA Guidance, Navigation and Control Conference and Exhibit , 2008, p. 6282

    2008

  3. [11]

    Full envelope missile longitudinal autopilot design using the state-dependent riccati equation method,

    C. Mracek, J. Cloutier, J. Cloutier, and C. Mracek, “Full envelope missile longitudinal autopilot design using the state-dependent riccati equation method,” in Guidance, Navigation, and Control Conference , 1997

    1997

  4. [12]

    SDRE autopilot for dual controlled missiles,

    C. P. Mracek, “SDRE autopilot for dual controlled missiles,” IFAC Proceedings Volumes, vol. 40, no. 7, pp. 750–755, 2007

    2007

  5. [13]

    Design and flight test of a robust autopilot for the iris- t air-to-air missile,

    H. Buschek, “Design and flight test of a robust autopilot for the iris- t air-to-air missile,” Control Engineering Practice , vol. 11, no. 5, pp. 551–558, 2003

    2003

  6. [14]

    Augmented three-loop autopilot structure based on mixed-sensitivity H∞ optimization,

    J.-H. Kim and I. H. Whang, “Augmented three-loop autopilot structure based on mixed-sensitivity H∞ optimization,” Journal of Guidance, Control, and Dynamics , vol. 41, no. 3, pp. 751–756, 2017

    2017

  7. [15]

    Connections between linear and nonlinear missile autopilots via three-loop topology,

    C.-H. Lee, B.-E. Jun, and J.-I. Lee, “Connections between linear and nonlinear missile autopilots via three-loop topology,” Journal of Guidance, Control, and Dynamics, vol. 39, no. 6, pp. 1426–1432, 2016

    2016

  8. [16]

    Introducing computational guidance and control,

    P. Lu, “Introducing computational guidance and control,” Journal of Guidance, Control, and Dynamics , vol. 40, no. 2, pp. 193–193, 2017

    2017

  9. [17]

    Predictive functional control-based missile autopilot design,

    W.-Q. Tang and Y .-L. Cai, “Predictive functional control-based missile autopilot design,” Journal of Guidance, Control, and Dynamics, vol. 35, no. 5, pp. 1450–1455, 2012

    2012

  10. [18]

    Nonlinear model predictive missile control with a stabilising terminal constraint,

    V . Bachtiar, T. M ¨uhlpfordt, W. Moase, T. Faulwasser, R. Findeisen, and C. Manzie, “Nonlinear model predictive missile control with a stabilising terminal constraint,” IFAC Proceedings Volumes, vol. 47, no. 3, pp. 457– 462, 2014

    2014

  11. [19]

    Nonlinear model-predictive integrated missile control and its multiobjective tuning,

    V . Bachtiar, C. Manzie, and E. C. Kerrigan, “Nonlinear model-predictive integrated missile control and its multiobjective tuning,” Journal of Guidance, Control, and Dynamics, vol. 40, no. 11, pp. 2961–2970, 2017

    2017

  12. [20]

    Control system optimization using genetic algorithms,

    K. Krishnakumar and D. E. Goldberg, “Control system optimization using genetic algorithms,” Journal of Guidance, Control, and Dynamics, vol. 15, no. 3, pp. 735–740, 1992

    1992

  13. [21]

    Multivariable con- troller design for aircraft longitudinal autopilot based on particle swarm optimization algorithm,

    B. Karimi, I. Saboori, and M. Lotfi-Forushani, “Multivariable con- troller design for aircraft longitudinal autopilot based on particle swarm optimization algorithm,” in 2011 IEEE International Conference on Computational Intelligence for Measurement Systems and Applications (C...

    2011

  14. [22]

    Adaptive critic nonlinear robust control: A survey,

    D. Wang, H. He, and D. Liu, “Adaptive critic nonlinear robust control: A survey,”IEEE Transactions on Cybernetics, vol. 47, no. 10, pp. 3429– 3451, 2017

    2017

  15. [23]

    Transforming cooling optimization for green data center via deep reinforcement learning,

    Y . Li, Y . Wen, K. Guan, and D. Tao, “Transforming cooling optimization for green data center via deep reinforcement learning,” IEEE Transac- tions on Cybernetics , 2019

    2019

  16. [24]

    Cooperative deep reinforcement learning for large-scale traffic grid signal control,

    T. Tan, F. Bao, Y . Deng, A. Jin, Q. Dai, and J. Wang, “Cooperative deep reinforcement learning for large-scale traffic grid signal control,” IEEE Transactions on Cybernetics , 2019

    2019

  17. [25]

    Reinforcement learning for uav attitude control,

    W. Koch, R. Mancuso, R. West, and A. Bestavros, “Reinforcement learning for uav attitude control,” ACM Transactions on Cyber-Physical Systems, vol. 3, no. 2, pp. 1–21, Feb 2019. [Online]. Available: http://dx.doi.org/10.1145/3301273

    2019 doi

  18. [26]

    Parameterized batch reinforcement learning for longitudinal control of autonomous land ve- hicles,

    Z. Huang, X. Xu, H. He, J. Tan, and Z. Sun, “Parameterized batch reinforcement learning for longitudinal control of autonomous land ve- hicles,” IEEE Transactions on Systems, Man, and Cybernetics: Systems , vol. 49, no. 4, pp. 730–741, 2017

    2017

  19. [27]

    Adaptive partial rein- forcement learning neural network-based tracking control for wheeled mobile robotic systems,

    L. Ding, S. Li, H. Gao, C. Chen, and Z. Deng, “Adaptive partial rein- forcement learning neural network-based tracking control for wheeled mobile robotic systems,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2018

    2018

  20. [28]

    Adaptive neural network control of auvs with control input nonlinearities using reinforcement learning,

    R. Cui, C. Yang, Y . Li, and S. Sharma, “Adaptive neural network control of auvs with control input nonlinearities using reinforcement learning,” IEEE Transactions on Systems, Man, and Cybernetics: Systems , vol. 47, no. 6, pp. 1019–1029, 2017

    2017

  21. [29]

    Online adaptive critic flight control,

    S. Ferrari and R. F. Stengel, “Online adaptive critic flight control,” Journal of Guidance, Control, and Dynamics , vol. 27, no. 5, pp. 777– 786, 2004

    2004

  22. [30]

    Helicopter trimming and tracking control using direct neural dynamic programming,

    R. Enns and J. Si, “Helicopter trimming and tracking control using direct neural dynamic programming,” IEEE Transactions on Neural networks , vol. 14, no. 4, pp. 929–939, 2003. 13 0 1 2 3 4 5 6 -20 0 20 40 60 80 100 120 (a) h = 6km 0 1 2 3 4 5 6 -20 0 20 40 60 80 100 120 140 (...

    2003

  23. [31]

    Incremental model based online dual heuristic programming for nonlinear adaptive control,

    Y . Zhou, E.-J. van Kampen, and Q. P. Chu, “Incremental model based online dual heuristic programming for nonlinear adaptive control,” Control Engineering Practice , vol. 73, pp. 13–25, 2018

    2018

  24. [32]

    Deterministic policy gradient with integral compensator for robust quadrotor control,

    Y . Wang, J. Sun, H. He, and C. Sun, “Deterministic policy gradient with integral compensator for robust quadrotor control,” IEEE Transactions on Systems, Man, and Cybernetics: Systems , 2019

    2019

  25. [33]

    Morphing control of a new bionic morphing uav with deep reinforcement learning,

    D. Xu, Z. Hui, Y . Liu, and G. Chen, “Morphing control of a new bionic morphing uav with deep reinforcement learning,”Aerospace Science and Technology, 2019

    2019

  26. [34]

    Depth control of model-free auvs via reinforcement learning,

    H. Wu, S. Song, K. You, and C. Wu, “Depth control of model-free auvs via reinforcement learning,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 49, no. 12, pp. 2499–2510, 2018

    2018

  27. [35]

    Using background knowledge to speed reinforcement learning in physical agents,

    D. Shapiro, P. Langley, and R. Shachter, “Using background knowledge to speed reinforcement learning in physical agents,” in Proceedings of the fifth international conference on Autonomous agents, 2001, pp. 254– 261

    2001

  28. [36]

    Deep reinforcement learning with prior knowledge,

    T. Chen, “Deep reinforcement learning with prior knowledge,” Master’s thesis, Pittsburgh, PA, May 2019

    2019

  29. [37]

    Knowledge matters: Importance of prior information for optimization,

    C ¸ . G¨ulc ¸ehre and Y . Bengio, “Knowledge matters: Importance of prior information for optimization,” The Journal of Machine Learning Re- search, vol. 17, no. 1, pp. 226–257, 2016

    2016

  30. [38]

    The value of prior knowledge in machine learning of complex network systems,

    D. Ferranti, D. Krane, and D. Craft, “The value of prior knowledge in machine learning of complex network systems,” Bioinformatics, vol. 33, no. 22, pp. 3610–3618, 2017

    2017

  31. [39]

    Using prior knowledge to improve reinforcement learning in mobile robotics,

    D. L. Moreno, C. V . Regueiro, R. Iglesias, and S. Barro, “Using prior knowledge to improve reinforcement learning in mobile robotics,” Proc. Towards Autonomous Robotics Systems. Univ. of Essex, UK , 2004

    2004

  32. [40]

    Continuous control with deep reinforcement learning,

    T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y . Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971 , 2015

    2015 arXiv

  33. [41]

    Low-frequency learning and fast adaptation in model reference adaptive control,

    T. Yucelen and W. M. Haddad, “Low-frequency learning and fast adaptation in model reference adaptive control,” IEEE Transactions on Automatic Control, vol. 58, no. 4, pp. 1080–1085, 2012

    2012

  34. [42]

    Connections between adaptive control and optimization in machine learning,

    J. E. Gaudio, T. E. Gibson, A. M. Annaswamy, M. A. Bolender, and E. Lavretsky, “Connections between adaptive control and optimization in machine learning,” arXiv preprint arXiv:1904.05856 , 2019

    1904 arXiv

  35. [43]

    Nonsmooth H∞ synthesis,

    P. Apkarian and D. Noll, “Nonsmooth H∞ synthesis,” IEEE Transac- tions on Automatic Control , vol. 51, no. 1, pp. 71–86, 2006

    2006

  36. [44]

    Parametric robust structured control design,

    P. Apkarian, M. N. Dao, and D. Noll, “Parametric robust structured control design,” IEEE Transactions on Automatic Control, vol. 60, no. 7, pp. 1857–1869, 2015

    2015

  37. [45]

    Effect of missile config- uration and inertial measurement unit location on autopilot response,

    N. Stein, H. Weiss, G. Hexner, and I. Rusnak, “Effect of missile config- uration and inertial measurement unit location on autopilot response,” Journal of Guidance, Control, and Dynamics , pp. 2740–2745, 2016

    2016

  38. [46]

    Missile longitudinal autopilots: comparison of multiple three loop topologies,

    C. Mracek and D. Ridgely, “Missile longitudinal autopilots: comparison of multiple three loop topologies,” in AIAA guidance, navigation, and control conference and exhibit , 2005

    2005

  39. [47]

    Bench- marking deep reinforcement learning for continuous control,

    Y . Duan, X. Chen, R. Houthooft, J. Schulman, and P. Abbeel, “Bench- marking deep reinforcement learning for continuous control,” in Inter- national Conference on Machine Learning , 2016, pp. 1329–1338

    2016

  40. [48]

    Deep reinforcement learning that matters,

    P. Henderson, R. Islam, P. Bachman, J. Pineau, D. Precup, and D. Meger, “Deep reinforcement learning that matters,” in Thirty-Second AAAI Conference on Artificial Intelligence , 2018

    2018

  41. [49]

    Reproducibil- ity of benchmarked deep reinforcement learning tasks for continuous control,

    R. Islam, P. Henderson, M. Gomrokchi, and D. Precup, “Reproducibil- ity of benchmarked deep reinforcement learning tasks for continuous control,” arXiv preprint arXiv:1708.04133 , 2017. Hyo-Sang Shin received his BSc on aerospace engineering from Pusan National University in 2...

    1995 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.