Pith. sign in

REVIEW 4 major objections 6 minor 30 references

Adviser-Actor-Critic: Eliminating Steady-State Error in Reinforcement Learning Control

T0 review · 4 major / 6 minor · reviewed 2026-08-09 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A PID controller acting as an 'adviser' can erase steady-state error in reinforcement learning control, letting actors hit precise goals.

desk verdict The paper's central proof is broken in two places, so the 'eliminating' claim is unsupported, but the empirical recipe and real quadcopter demo make it a salvageable submission. read the letter →

arxiv 2502.02265 v1 pith:2EB2M2PK submitted 2025-02-04 cs.LG cs.AI

Donghe Chen , Yubin Peng , Tengjie Zheng , Han Wang , Chaoran Qu , Lin Cheng This is my paper · ORCID
classification cs.LGcs.AI
keywords reinforcementlearningsteady-stateerrorPIDcontrolgoal-conditionedactor-criticfakegoalsadvisermechanismquadcopter
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that reinforcement learning agents can achieve high-precision goal tracking by having a PID controller act as an 'adviser' that feeds a synthetic error signal into the actor's observations. The central claim is that this fake-goal mechanism reduces steady-state error far below what standard actor-critic methods achieve, without requiring a system model. The authors support the claim with experiments on a mass-spring-damper, a robotic arm, and a real quadcopter, where adviser-augmented SAC and SAC+HER show lower steady-state error and higher cumulative reward. If the claim holds, the framework offers a simple retrofit for existing actor-critic implementations to reach precise setpoints in robot control.

What carries the argument

The Adviser is the load-bearing component: it is a PID controller that takes the tracking error $e = g_d - g_a$ and outputs a fake error $\epsilon = -K_p e - K_i \int e \, d\tau - K_d \frac{de}{dt}$, appended to the actor's observation. The actor learns to act on this augmented input, which biases the policy toward the fake goal; the RL loop then corrects the bias online. Stability of the adviser is checked with the Routh–Hurwitz criterion on a per-dimension second-order model, giving the conditions $K'_d > 0$, $K_i > 0$, and $K'_p K'_d > K_i$. The error-reduction property is derived from the Laplace-domain equation $A(s)E(s) = B U(s) + D(s)$, leading to the contraction condition $\rho(I - B) < 1$ for the steady-state error.

What would settle it

Take the mass-spring-damper or quadcopter plant, measure its DC-gain matrix $A(0)$, and apply the adviser's fake-goal update while holding a constant setpoint. If $A(0)$ is not the identity or $\rho(I - B)$ is not below 1, the steady-state error should fail to contract on each iteration, directly contradicting the paper's Eq. (19) guarantee.

Watch

Extended reading notes

Core claim

The central discovery is that a PID controller used as an adviser can eliminate the residual steady-state error that actor-critic policies leave in goal-conditioned tasks. The adviser computes an error signal $\epsilon$ from the current goal error plus proportional, integral, and derivative terms, and feeds $\epsilon$ to the actor; the actor is thereby steered toward a 'fake goal' that compensates for its systematic bias. The paper formalizes the error dynamics of this setup and derives that, under stability and command-tracking conditions, the new steady-state error equals $(I - B)$ times the previous one, so the error contracts when the spectral radius $\rho(I - B)$ is less than 1. In experiments, the adviser lowers steady-state error and raises cumulative reward for both SAC and SAC+HER across three environments, and a real quadcopter reaches a 5° roll target that the unaided policy misses.

Load-bearing premise

For the claimed error reduction to hold, the plant must have no steady-state distortion at zero frequency and each axis must act independently; real coupled robots satisfy neither automatically.

Editorial extensions

If this is right

  • With the adviser active, both SAC and SAC+HER show lower steady-state error and higher cumulative reward across all three test environments.
  • Training with the adviser active and evaluating with different gains still helps, meaning the adviser's parameters can be readjusted at deployment without retraining.
  • The method transfers to a physical quadcopter: adviser-guided roll control reaches the 5° target that the unaided policy misses.
  • Because the RL side of the framework is model-free, the adviser does not require a dynamics model of the plant, unlike MPC.
  • The adviser can be combined with replay strategies such as HER, so it is an additive modification rather than a replacement for existing goal-conditioned RL tools.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's proof of contraction requires $A(0)=I$, so for plants with a non-identity DC gain the same 'fake goal' update could fail or even increase error; a direct test is to measure the plant's DC-gain matrix before applying AAC.
  • Because the adviser acts only on the error, AAC should also reduce steady-state error caused by unmodeled disturbances, not just actor bias, as long as the disturbance enters the error dynamics the same way.
  • A natural extension the authors leave implicit is to make the adviser gains learnable: a meta-controller could tune $K_p, K_i, K_d$ online, which would generalize the framework beyond the fixed-gain experiments reported here.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes Adviser-Actor-Critic (AAC), a framework that augments an actor-critic RL agent with a PID-based 'adviser' that generates fake goals and synthetic error signals. The authors claim this eliminates steady-state error in goal-conditioned control tasks, and support the claim with a Routh-Hurwitz stability analysis and an appendix proof that iterating fake-goal updates contracts the steady-state error. They evaluate AAC against SAC and SAC+HER on three environments—a mass-spring-damper system, a 3-DoF robotic arm, and a quadcopter—and also deploy the trained policy on a physical quadcopter.

Significance. If the central guarantee were valid, combining PID-based guidance with RL would be a practically useful way to improve precision in goal-conditioned control. The paper's strengths include a real-world quadcopter experiment and released code. However, the theoretical proof that fake goals eliminate steady-state error is invalid: it relies on an unstated A(0)=I assumption and reverses the standard spectral-radius-norm inequality. As a result, the title claim of 'eliminating' steady-state error is unsupported, and the contribution reduces to an empirical observation that a PID-style signal can improve RL performance—already expected from classical control theory.

major comments (4)
  1. [Appendix A, Eq. (15)] The derivation of ess,now = (I - B) ess,prev equates lim_{s->0} A(s)^{-1} B with B. This is valid only if A(0) = I or B has special structure; neither is stated. For the second-order model of Eq. (6), the scalar A(s) is s^2 - a1 s - a0, so A(0) = -a0, which is not 1 in general. Consequently the claimed contraction does not follow from the stated assumptions.
  2. [Appendix A, Eq. (18)] The inequality rho(I - B) >= ||I - B||_2 is reversed; the standard property is rho(M) <= ||M|| for every matrix norm. The chain in Eq. (19) therefore does not establish ||ess,now||_2 < ||ess,prev||_2, even under rho(I - B) < 1. A concrete counterexample is B = [[0.1, -10], [0, 0.1]], for which rho(I - B) = 0.9 but ||I - B||_2 is approximately 10.1.
  3. [Sec. 4.4 and Appendix A] The stability analysis and the fake-goal contraction argument assume the system dynamics are decoupled into independent second-order per-dimension equations. The paper asserts this decoupling but provides no justification for the MIMO environments (robotic arm, quadcopter) in which it is evaluated. Without a validated decoupling assumption, the theoretical result cannot be claimed to apply to the experimental systems.
  4. [Sec. 4.3 and Figure 2] The algorithmic mechanism is underspecified. The paper defines the adviser's synthetic error epsilon(t) in Eq. (5) but never states how this signal enters the actor's policy or how the fake goal is constructed from epsilon. The experimental section provides only parameter tables, not the policy update rule. This makes the method difficult to reproduce from the text and prevents the reader from determining whether the reported gains are due to the adviser mechanism or to other implementation details.
minor comments (6)
  1. [Abstract] The phrase 'Code are available' should be 'Code is available'.
  2. [Eqs. (15) and (19)] Equation (19) uses ess,new while Eq. (15) defines ess,now; the notation should be consistent.
  3. [Table 4] The table provides adviser hyperparameters only for the robotic arm; the corresponding settings for the mass-spring-damper and quadcopter experiments are missing.
  4. [Figure 7] The figure reports cumulative reward and steady-state error without error bars or multiple-seed statistics; the paper does not state how many random seeds were run.
  5. [Appendix A] The phrase 'command tracking' is not formally defined beyond a heuristic description; the paper should give a precise mathematical condition.
  6. [Figure 3 caption] The relation between the gains K'_p and K'_d used in Eq. (7) and the actual PID gains K_p, K_d, K_i in Eq. (5) is explained only in the caption of Figure 3 and not in the main text; this connection should be made explicit in the body of the paper.

Circularity Check

0 steps flagged · score 2.0 of 10

No significant circularity: the core gain comes from an imported PID controller and external benchmarks; Appendix A's proof errors are correctness issues, not circular reductions.

full rationale

The central mechanism is an explicit PID controller (Eq. 5) used as an Adviser; steady-state-error reduction is a classical, externally established property of integral control, so the performance gain is imported from control theory rather than manufactured by the paper's own definitions. The ablation in Figure 7 compares SAC/SAC+HER with and without the adviser on three external benchmark environments, and the real-quadcopter experiment is an independent deployment, so the headline claim is empirically grounded rather than a renamed fit. The Appendix A proof has two mathematical defects: Eq. 15 silently uses lim_{s->0} A(s)^{-1}B = B, which requires the unstated assumption A(0)=I, and Eq. 18 asserts rho(I-B) >= ||I-B||_2, the reverse of the standard spectral-radius bound. These defects invalidate the theorem as written, but they are correctness risks, not circularities: the proof does not assume the conclusion; it states conditions (stability, command tracking, rho(I-B)<1) and attempts to derive contraction. The only self-citations are two contextual references in the introduction (Cheng et al. 2023; Qu et al. 2024) from the same group; they are not load-bearing for the AAC derivation. No fitted parameter is renamed as a prediction, and no uniqueness theorem is imported from prior work. Hence no significant circularity; score 2 reflects only the minor, non-load-bearing self-citations.

Assumptions & free parameters 2 free parameters · 6 assumptions · 1 invented entities

The central claim depends on the error-dynamics model of Appendix A, which assumes per-dimension decoupling, a qualified actor with stability and command tracking, and the unstated A(0)=I condition; the contraction condition rho(I-B)<1 is assumed rather than derived. The PID gains are hand-set free parameters.

free parameters (2)
  • PID gains Kp, Ki, Kd (Adviser) = Kp=1.3, Ki=0.01, Kd=0.01 (train), Kp=1.3, Ki=0.1, Kd=0.1 (eval) on robotic arm; other environments not fully specified
    Manually chosen per environment (Table 4); no tuning rule is given, and the claimed error reduction depends on these values.
  • Reward function weights = e.g., mass-spring-damper: 1.0, 0.5, 0.1; arm: 1.0, 0.1, 0.1
    Hand-designed per environment; these weights shape the learned policy and affect the empirical comparison, though they are standard experimental design choices.
assumptions (6)
  • domain assumption Per-dimension decoupled second-order linear error dynamics (Eq. 6, Figure 3)
    Used for the Routh-Hurwitz analysis and for the fake-goal proof; general MIMO nonlinear systems do not satisfy this without justification.
  • ad hoc to paper A(0)=I in the error dynamics matrix of Appendix A
    Needed for ess,now=(I-B)ess,prev (Eq. 15) but never stated or justified. Without it the contraction result collapses.
  • domain assumption The actor is a 'qualified actor' with stability and command tracking (Appendix A, Eq. 14)
    The fake-goal proof assumes the learned policy is stable and tracks commands; the paper provides no mechanism guaranteeing this for a learned neural policy.
  • ad hoc to paper rho(I-B)<1, the contraction condition for the fake-goal update (Eq. 19)
    Assumed to hold; the paper does not show it follows from stability conditions (9) or any training procedure.
  • standard math Routh-Hurwitz criterion applied to a cubic characteristic polynomial (Eq. 8-9)
    Standard algebraic stability criterion, correctly applied for a linear third-order system.
  • ad hoc to paper The fake-goal input perturbation equals -ess,prev/s in the Laplace domain (Eq. 15)
    Assumes the adviser shifts the reference by exactly the previous steady-state error, a mechanism not precisely defined in the algorithm section.
invented entities (1)
  • Adviser (fake goal / synthetic error signal epsilon)
    purpose: Applies a PID transform to the tracking error and feeds the result to the Actor as a guiding signal, so the agent chases a 'fake goal' instead of the true one.
    The synthetic error is an internal algorithmic construct; the only evidence for its mechanism is the paper's own flawed Appendix A analysis and single-run experiments.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Adviser-Actor-Critic: Eliminating Steady-State Error in Reinforcement Learning Control." pith.science (2026). https://pith.science/paper/2EB2M2PK

@misc{pith2026250202265,
  author       = {Pith},
  title        = {Pith review of: Adviser-Actor-Critic: Eliminating Steady-State Error in Reinforcement Learning Control},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2EB2M2PK}},
  note         = {Machine review of arXiv:2502.02265}
}
read the original abstract

High-precision control tasks present substantial challenges for reinforcement learning (RL) algorithms, frequently resulting in suboptimal performance attributed to network approximation inaccuracies and inadequate sample quality.These issues are exacerbated when the task requires the agent to achieve a precise goal state, as is common in robotics and other real-world applications.We introduce Adviser-Actor-Critic (AAC), designed to address the precision control dilemma by combining the precision of feedback control theory with the adaptive learning capability of RL and featuring an Adviser that mentors the actor to refine control actions, thereby enhancing the precision of goal attainment.Finally, through benchmark tests, AAC outperformed standard RL algorithms in precision-critical, goal-conditioned tasks, demonstrating AAC's high precision, reliability, and robustness.Code are available at: https://anonymous.4open.science/r/Adviser-Actor-Critic-8AC5.

Figures

Figures reproduced from arXiv: 2502.02265 by the authors.

Figure 1
Figure 1. When a robot aims directly for a desired goal, control inaccuracies or model bias can prevent it from reaching the target precisely. However, by guiding the robot toward a strategically placed ”fake goal,” set by an adviser, it can effectively achieve the desired position [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Interact With Environment: The Adviser outputs a synthetic error ε to guide the Actor Neural Network’s decision-making, alongside the achieved goal ga and current observation s. This framework integrates deep learning with classical control theory, enabling adaptive policy optimization and enhancing performance in dynamic environments. Learn from Experience: The Critic Neural Network estimates the state-action value… view at source ↗
Figure 3
Figure 3. Within the dynamic system, PID controllers transform the actual error e into a fake error ε. For simplified stability analysis, the PID controller parameters and dynamic system parameters are combined, adjusting K′ p = Kp − a0 and K′ d = Kd − a1. εi represents the adjusted error, while di denotes disturbances in the i-th dimension, accounting for unmodeled dynamics. Notably, when Kp = 1, Ki = 0, and Kd = 0, the PID … view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Response of a Second-Order System to Command under Different PID Parameters: Analyzing Asymptotic, Marginal, and Instability Conditions our trainning procedure. Second, we compare the perfor￾mance of SAC/SAC-HER with and without adviser. Third, we show how advisers lea…
Figure 6
Figure 6. Figure 6: Illustration of how the adviser uses intermediate goals to guide the robot towards the desired target. Starting from position (0, 0), the robot aims for a point at (1.2, 1.2). Although the interme￾diary targets do not perfectly align with the final destination, they se…
Figure 5
Figure 5. Figure 5: Different environments: Position Control of a Mass￾Spring-Damper System (left), Robotics Arm Fetch (middle) and Velocity Control of a Quadcopter (right). We evaluate the AAC on three tasks, as detailed below. It is represented as Multi-Layer Perceptrons (MLPs) with Sig…
Figure 7
Figure 7. Figure 7: Performance comparison of four adviser configurations under SAC and SAC+HER across three environments. The adviser reduces steady-state errors and increases cumulative rewards, demonstrating enhanced stability and accelerated training. adviser’s impact on learning dyna…
Figure 8
Figure 8. Figure 8: Illustration of the quadcopter’s orientations. The quad￾copter, with a takeoff weight of 1.40 kg and an axle distance of 35.0 cm, uses a Pixhawk4 for low-level control and ROS for high￾level commands. Indoor flights utilize a motion capture system for precise positioni…
Figure 9
Figure 9. Figure 9: Comparison of roll angle control with and without an adviser, highlighting improved precision and efficiency. all performance. This advancement underscores the poten￾tial for developing more sophisticated and dependable UAV (Unmanned Aerial Vehicle) control systems for…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 19 canonical work pages

  1. [1]

    Hindsight experience replay

    Andrychowicz, M., Wolski, F., Ray, A., Schneider, J., Fong, R., Welinder, P., McGrew, B., Tobin, J., Pieter Abbeel, O., and Zaremba, W. Hindsight experience replay. Advances in neural information processing systems, 30, 2017

    work page 2017

  2. [2]

    P., Maghade, D., Sondkar, S., and Pawar, S

    Borase, R. P., Maghade, D., Sondkar, S., and Pawar, S. A review of pid control, tuning methods and applications. International Journal of Dynamics and Control, 9: 0 818--827, 2021

    work page 2021

  3. [3]

    Exploration by random network distillation

    Burda, Y., Edwards, H., Storkey, A., and Klimov, O. Exploration by random network distillation. In Seventh International Conference on Learning Representations, pp.\ 1--17, 2019

    work page 2019

  4. [4]

    Reinforcement learning for control: Performance, stability, and deep approximators

    Bu s oniu, L., De Bruin, T., Toli \'c , D., Kober, J., and Palunko, I. Reinforcement learning for control: Performance, stability, and deep approximators. Annual Reviews in Control, 46: 0 8--28, 2018

    work page 2018

  5. [5]

    Learning sim-to-real dense object descriptors for robotic manipulation

    Cao, H.-G., Zeng, W., and Wu, I.-C. Learning sim-to-real dense object descriptors for robotic manipulation. In 2023 IEEE International Conference on Robotics and Automation (ICRA), pp.\ 9501--9507. IEEE, 2023

    work page 2023

  6. [6]

    Neural-network-based nonlinear optimal terminal guidance with impact angle constraints

    Cheng, L., Wang, H., Gong, S., and Huang, X. Neural-network-based nonlinear optimal terminal guidance with impact angle constraints. IEEE Transactions on Aerospace and Electronic Systems, 2023

    work page 2023

  7. [7]

    Darby, M. L. and Nikolaou, M. Mpc: Current practice and challenges. Control Engineering Practice, 20 0 (4): 0 328--342, 2012

    work page 2012

  8. [8]

    Addressing function approximation error in actor-critic methods

    Fujimoto, S., Hoof, H., and Meger, D. Addressing function approximation error in actor-critic methods. In International conference on machine learning, pp.\ 1587--1596. PMLR, 2018

    2018

Show all 30 references
  1. [9]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor

    Haarnoja, T., Zhou, A., Abbeel, P., and Levine, S. Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor. In International conference on machine learning, pp.\ 1861--1870. PMLR, 2018

    2018

  2. [10]

    Learning to utilize shaping rewards: A new approach of reward shaping

    Hu, Y., Wang, W., Jia, H., Wang, Y., Chen, Y., Hao, J., Wu, F., and Fan, C. Learning to utilize shaping rewards: A new approach of reward shaping. Advances in Neural Information Processing Systems, 33: 0 15931--15941, 2020

    2020

  3. [11]

    D., Tsounis, V., Hwangbo, J., Bodie, K., Fankhauser, P., Bloesch, M., et al

    Hutter, M., Gehring, C., Jud, D., Lauber, A., Bellicoso, C. D., Tsounis, V., Hwangbo, J., Bodie, K., Fankhauser, P., Bloesch, M., et al. Anymal-a highly mobile and dynamic quadrupedal robot. In 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp...

    2016

  4. [12]

    R., Sobh, I., Talpaert, V., Mannion, P., Al Sallab, A

    Kiran, B. R., Sobh, I., Talpaert, V., Mannion, P., Al Sallab, A. A., Yogamani, S., and P \'e rez, P. Deep reinforcement learning for autonomous driving: A survey. IEEE Transactions on Intelligent Transportation Systems, 23 0 (6): 0 4909--4926, 2021

    2021

  5. [13]

    H., and Chong, G

    Li, Y., Ang, K. H., and Chong, G. C. Pid control system analysis and design. IEEE Control Systems Magazine, 26 0 (1): 0 32--41, 2006

    2006

  6. [14]

    Continuous control with deep reinforcement learning

    Lillicrap, T. Continuous control with deep reinforcement learning. arXiv preprint arXiv:1509.02971, 2015

    2015 arXiv

  7. [15]

    A review of industrial mimo decoupling control

    Liu, L., Tian, S., Xue, D., Zhang, T., Chen, Y., and Zhang, S. A review of industrial mimo decoupling control. International Journal of Control, Automation and Systems, 17 0 (5): 0 1246--1254, 2019

    2019

  8. [16]

    Goal-conditioned reinforcement learning: Problems and solutions

    Liu, M., Zhu, M., and Zhang, W. Goal-conditioned reinforcement learning: Problems and solutions. arXiv preprint arXiv:2201.08299, 2022

    2022 arXiv

  9. [17]

    A., Veness, J., Bellemare, M

    Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., Graves, A., Riedmiller, M., Fidjeland, A. K., Ostrovski, G., et al. Human-level control through deep reinforcement learning. nature, 518 0 (7540): 0 529--533, 2015

    2015

  10. [18]

    Goal-directed planning via hindsight experience replay

    Moro, L., Likmeta, A., Prati, E., Restelli, M., et al. Goal-directed planning via hindsight experience replay. In 10th International Conference on Learning Representations, ICLR 2022, pp.\ 1--16, 2022

    2022

  11. [19]

    A., and Darrell, T

    Pathak, D., Agrawal, P., Efros, A. A., and Darrell, T. Curiosity-driven exploration by self-supervised prediction. In International conference on machine learning, pp.\ 2778--2787. PMLR, 2017

    2017

  12. [20]

    Dynamic-matching adaptive sliding mode control for hypersonic vehicles

    Qu, C., Cheng, L., Gong, S., and Huang, X. Dynamic-matching adaptive sliding mode control for hypersonic vehicles. Aerospace Science and Technology, 149: 0 109159, 2024

    2024

  13. [21]

    Discovering blind spots in reinforcement learning

    Ramakrishnan, R., Kamar, E., Dey, D., Shah, J., and Horvitz, E. Discovering blind spots in reinforcement learning. In Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems, pp.\ 1017--1025, 2018

    2018

  14. [22]

    Proximal policy optimization algorithms

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  15. [23]

    Deterministic policy gradient algorithms

    Silver, D., Lever, G., Heess, N., Degris, T., Wierstra, D., and Riedmiller, M. Deterministic policy gradient algorithms. In International conference on machine learning, pp.\ 387--395. Pmlr, 2014

    2014

  16. [24]

    Action robust reinforcement learning and applications in continuous control

    Tessler, C., Efroni, Y., and Mannor, S. Action robust reinforcement learning and applications in continuous control. In International Conference on Machine Learning, pp.\ 6215--6224. PMLR, 2019

    2019

  17. [25]

    D., Michi, A., Chervonyi, Y., Davies, I., Paduraru, C., Lazic, N., Felici, F., Ewalds, T., Donner, C., Galperti, C., et al

    Tracey, B. D., Michi, A., Chervonyi, Y., Davies, I., Paduraru, C., Lazic, N., Felici, F., Ewalds, T., Donner, C., Galperti, C., et al. Towards practical reinforcement learning for tokamak magnetic control. Fusion Engineering and Design, 200: 0 114161, 2024

    2024

  18. [26]

    A brief survey on nonlinear control using adaptive dynamic programming under engineering-oriented complexities

    Zhang, Y., Zou, L., Liu, Y., Ding, D., and Hu, J. A brief survey on nonlinear control using adaptive dynamic programming under engineering-oriented complexities. International Journal of Systems Science, 54 0 (8): 0 1855--1872, 2023

    2023

  19. [27]

    P., Qingqing, L., and Westerlund, T

    Zhao, W., Queralta, J. P., Qingqing, L., and Westerlund, T. Towards closing the sim-to-real gap in collaborative multi-robot deep reinforcement learning. In 2020 5th International conference on robotics and automation engineering (ICRAE), pp.\ 7--12. IEEE, 2020 a

    2020

  20. [28]

    P., and Westerlund, T

    Zhao, W., Queralta, J. P., and Westerlund, T. Sim-to-real transfer in deep reinforcement learning for robotics: a survey. In 2020 IEEE symposium series on computational intelligence (SSCI), pp.\ 737--744. IEEE, 2020 b

    2020

  21. [29]

    A comprehensive survey on transfer learning

    Zhuang, F., Qi, Z., Duan, K., Xi, D., Zhu, Y., Zhu, H., Xiong, H., and He, Q. A comprehensive survey on transfer learning. Proceedings of the IEEE, 109 0 (1): 0 43--76, 2020

    2020

  22. [30]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 9, 2026 · model on record in the stance chip above.