Pith. sign in

REVIEW 4 major objections 5 minor 34 references

Knowledge-Distilled End-to-End Reinforcement Learning for Smooth 6-DOF Thrust Control and Rapid Adaptation to Ocean Currents in Remotely Operated Vehicles

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read An end-to-end reinforcement learning controller can hold a six-degree-of-freedom remotely operated vehicle on station under time-varying ocean currents more accurately, faster, more energy-efficiently, and with smoother thrust than a…

desk verdict A competent RL-for-ROV paper with a narrow but real architectural novelty, whose headline superiority claim is held hostage by an undocumented P-PID baseline and reward-overlapping energy metrics. read the letter →

arxiv 2608.08598 v1 pith:VQATCP6C submitted 2026-08-09 cs.RO

Tiankuang Wen , Huiping Li , Gang Liu , Yong Jiang This is my paper · ORCID
classification cs.RO
keywords deepreinforcementlearningremotelyoperatedvehicles6-DOFcontrolthrustsmoothingoceancurrentadaptationknowledgedistillationproximalpolicyoptimizationstation-keeping
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

TSRCA-PPO is a proximal-policy-optimization controller that maps onboard observations directly to eight thruster commands, bundling motion control and thrust allocation into one learned policy. The paper's central claim is that this end-to-end approach meets a combination of requirements that earlier reinforcement-learning ROV controllers have not met at once: low steady-state position and attitude error, fast settling, low energy use, and smooth thrust under time-varying ocean currents. The method first trains a teacher policy with privileged information—true physical parameters, true dynamic states, motor speeds, and current conditions—then distills the teacher's latent current-awareness into a student policy that uses only a history of ordinary onboard observations. In simulation, the student policy reduces the position error, attitude error, settling time, energy index, and thrust-smoothness index to 42.7%, 76.5%, 10.6%, 93.5%, and 15.9% of the corresponding cascaded P-PID values, and raises the success rate from 72.8% to 100.0%.

What carries the argument

The load-bearing object is the two-stage distillation pipeline with a privileged multi-encoder. Stage I trains a PPO teacher whose actor encodes the action-observation pair and three separately encoded privileged groups—true static parameters, true dynamic states including actual motor rotational speeds, and true current parameters—into a low-dimensional latent feature $z^{\mathrm{priv}}_t$; the critic receives the full concatenated privileged state. Stage II freezes the base policy and trains a GRU-based adaptation module, using DAgger-style on-policy data collection, to predict the latent feature from a history buffer $H_t = [(\tilde{o}_{t-L}, a_{t-L}), \ldots, (\tilde{o}_{t-1}, a_{t-1})]$ by minimizing a three-branch regression loss. This mechanism decouples optimal-policy learning from implicit state estimation: the deployable student infers current conditions and internal parameters from proprioceptive history rather than requiring a current sensor.

What would settle it

Run the same 1,000-episode station-keeping evaluation against a P-PID baseline whose gains are re-tuned under the same settling criterion and augmented with ocean-current feedforward and anti-windup; if its settling time and thrust-smoothness index approach TSRCA-PPO's values, the claimed superiority would be shown to depend on the baseline's construction rather than on the learned controller.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a carefully shaped reward plus a privileged multi-encoder lets reinforcement learning produce near-optimal station-keeping for an overactuated 6-DOF ROV under currents the policy has not been designed around. The reward's velocity-guidance terms project the vehicle's velocity onto the direction of its position and attitude error, acting as an implicit braking signal that suppresses overshoot and oscillation near the target. The privileged multi-encoder splits static parameters, dynamic states, and current flow into separate latent branches; ablation shows the dynamic-state branch is the main contributor to thrust smoothness, while the current-flow branch mainly shortens settling time. After distillation, the student estimates the teacher's latent feature from a 30-step history of observations and actions, and runs in 4.407 ms per inference step, well below the 16 ms control period.

Load-bearing premise

The whole comparison rests on the assumption that the cascaded P-PID controller is a well-tuned representative of conventional practice; the paper does not report its gains, tuning procedure, or whether it compensates for currents, so the reported margins could shrink if the baseline were made stronger.

Editorial extensions

If this is right

  • The student policy's 4.407 ms inference latency fits inside the 16 ms control period, so the method can run in real time rather than only as an offline benchmark.
  • Because removing the dynamic-state branch raises the force-smoothness index from 0.26 to 0.61 in Stage I and roughly doubles it in Stage II, accurate onboard state estimation is the main lever that buys thrust smoothness.
  • Because removing the current-flow branch primarily lengthens settling time, rapid current adaptation is concentrated in the flow-latent branch rather than spread across the whole policy.
  • The distilled student achieves 100.0% success under the 0.02 m and 2-degree settling criterion while using only observation histories, so the privileged information does not need to be measured at deployment.
  • A stepwise current-reversal test shows the policy recovers attitude shortly after each 90-degree current-direction change, indicating the flow latent is being tracked online rather than merely fitting a static offset.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension would be to benchmark the same student policy against a P-PID baseline augmented with ocean-current feedforward and anti-windup; the paper does not report the baseline's gains, tuning procedure, or such augmentations, so it is an open question which margins survive against a stronger conventional controller.
  • If dynamic-state latent information is the main driver of smoothness, then replacing the GRU adaptation module with a lightweight recursive estimator of motor RPM and acceleration could bring similar smoothness to platforms without the full distillation pipeline.
  • The energy comparison uses a cubic-RPM proxy rather than measured electrical power; instrumenting thrusters to log true power in a hardware test would show whether the 6.4% energy saving is real or an artifact of the proxy.
  • The method's success in simulation suggests a natural next step of zero-shot transfer to a physical ROV in a current basin, where the student's reliance on observation history could be stress-tested under sensor dropout and unmodeled waves.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes TSRCA-PPO, a two-stage reinforcement-learning controller for 6-DOF station keeping of an overactuated ROV. Stage I trains a PPO teacher policy with privileged information split into static, dynamic, and current-flow encoders; Stage II distills the resulting latent representation into a deployable student policy using a GRU-based adaptation module trained with DAgger-style data aggregation. The reward includes terminal pose terms, velocity-guidance terms, an energy proxy, and an action-smoothness term. Evaluation is performed in a MarineGym-style simulator with domain randomization and Gauss-Markov ocean currents. The central claim is that TSRCA-PPO consistently outperforms a conventional cascaded P-PID controller on steady-state position error, steady-state attitude error, settling time, success rate, energy index, and thrust-smoothness index, with the specific reductions listed in the abstract and conclusions.

Significance. The paper addresses a relevant problem and contains a substantial simulation study: 1,000 evaluation episodes per configuration, multiple metrics, ablations over the architecture components, and a latency measurement (4.407 ms per inference step) showing real-time feasibility relative to the 16 ms control period. The idea of structuring privileged information into separate encoders and distilling it with a temporal adaptation module is sensible, and the ablations give useful insight, such as the dynamic-state branch being the main contributor to action smoothness. If the comparative claims are supported, the method would be a useful contribution to RL-based ROV control. However, the headline comparison is weakened by the undocumented single baseline and by the overlap between some evaluation metrics and the training objective.

major comments (4)
  1. [Section 5.2, Table 8] The single comparator is a 'conventional cascaded P-PID' whose gains, tuning procedure, anti-windup, and feedforward terms are not reported. Every headline percentage in the abstract and conclusion is relative to this controller. Because the P-PID has no current feedforward while the evaluation injects stepwise current changes (Fig. 9), and because its success rate is only 72.8%, the reported margins (e.g., settling time 28.55 s vs. 3.04 s) may reflect an under-tuned baseline rather than an inherent advantage of the proposed method. I ask the authors to report the controller parameters and tuning procedure, and to add at least one better-tuned or current-feedforward baseline to substantiate the superiority claim.
  2. [Section 5.1, settling-time definition; Table 8] The text states that 'the settling time is computed only over successful episodes.' P-PID succeeds in only 72.8% of episodes while TSRCA-PPO succeeds in 100%. The reported mean settling time of 28.55 s for P-PID is therefore a conditional mean over the successful subset, whereas 3.04 s is effectively unconditional; comparing these two numbers is not apples-to-apples. Please report the settling-time statistic over all episodes, treating unsuccessful episodes as censored at the horizon, or report quantiles in addition to the conditional mean.
  3. [Equations (37)-(39) and (43)-(44)] The energy index J_E in Eq. (43) is exactly the accumulated per-step energy proxy E_t that enters the reward through Eq. (38), and the thrust-smoothness metric J_FS, although formally a second-order force difference, is directly driven by the action-smoothness reward in Eq. (39) because the policy is trained to minimize consecutive action changes. Consequently, the reported reductions in energy and smoothness are partly a direct consequence of the training objective and are not independent measurements of the controller's intrinsic efficiency or smoothness. Please state this overlap explicitly and, if possible, compare against a baseline that is tuned or optimized on the same objective, or use genuinely independent metrics.
  4. [Section 6, conclusion] The conclusion states that 'Ablation studies validate the effectiveness and rationality of the proposed reward function,' but Section 5.1 explicitly ablates only the architecture (static/dynamic/current branches, single encoder, uncompressed latent) while keeping the reward function fixed. No ablation of the reward terms (e.g., velocity guidance, energy, smoothness, terminal terms) is presented. Either add such an ablation or temper the conclusion to refer only to the architecture and privileged-information design.
minor comments (5)
  1. [Equation (32)] The expression '180/π 2 arccos(|⟨q_t,q_d⟩|)' appears to be missing a multiplication sign; it should presumably be (180/π) × 2 arccos(|⟨q_t,q_d⟩|) or similar.
  2. [Equation (36)] The notation 'sgn(q_e,w,t) q_e,v,t' is ambiguous: it is unclear whether the sign is applied to the scalar quaternion component, to the vector part, or to each component of the vector part. Please define this notation precisely.
  3. [Figure 10 and Table 8] The single-trajectory force-smoothness values in Fig. 10 (0.23 for TSRCA-PPO, 0.92 for P-PID) differ from the mean values in Table 8 (0.27 and 1.70); the caption and text should state clearly that the figure shows one representative example, not the average over the 1,000 evaluation episodes.
  4. [Tables 6 and 7, inference latency] The PPO and TQC baselines have inference latencies of 0.86 ms and 0.36 ms, respectively, which are substantially lower than the 4.41 ms of the proposed student policy. The paper should comment on this latency increase as a cost of the adaptation module, even though 4.41 ms is still below the 16 ms control period.
  5. [Section 5.1, latent-feature MSE] The phrase 'quantified by theZmean squared error' appears to be a typesetting artifact; please rephrase it as 'quantified by the mean squared error of the latent feature' or similar.

Circularity Check

1 steps flagged · score 6.0 of 10

The energy-index comparison is partly forced by construction because the evaluation metric is the same per-step proxy used in the reward; the other headline metrics remain independent.

  1. self definitional [Section 4.4 (Eqs. 37-38) and Section 5.1 (Eq. 43)]
    "The reward is formulated as an exponential function to reduce the energy-consumption proxy during motion: re,t = 12 exp(−2.5×10−6 Et). ... The energy consumption proxy is obtained by summing the per-step propulsion-energy proxy over time: JE = N∑ t=1 Et = N∑ t=1 8∑ i=1 | ni(t)/60 |^3."

    JE (Eq. 43) is the accumulated sum of Et, the exact same per-step proxy that appears in the energy reward re,t (Eq. 38). PPO maximizes the discounted sum of rewards including re,t, so minimizing JE is part of the training objective. Reporting that TSRCA-PPO reduces JE to 93.5% of the P-PID value is therefore a check that the reward optimization succeeded, not an independent measurement of energy efficiency. The force-smoothness metric is not identical to the action-difference smoothness reward, and position/attitude error and settling time are genuine control objectives, so the circularity is partial and limited to the energy-index claim.

full rationale

The only clear by-construction reduction is the energy index: Eq. 43 sums the per-step energy proxy Et, and Eq. 38 places that same Et inside the reward. Consequently the reported energy saving is substantially a consequence of optimizing the reward. The thrust-smoothness metric (Eq. 44) uses a second-order force difference and is not algebraically the same as the action-difference reward (Eq. 39), so I do not flag it as forced. The steady-state position/attitude errors and settling time are tied to reward terms, but those terms express the control task itself rather than a hidden fitted input. The P-PID baseline is undocumented in tuning and gains, which undermines the strength of the superiority claim, but that is a correctness risk, not circularity. The ablations and distillation evaluation (MSE_Z) compare the student against the teacher latent that the student was trained to match, yet that is an internal fidelity check rather than the paper's central external claim. Overall, one of the five headline metrics reduces to the training objective by definition, giving partial circularity, while the rest of the comparison retains independent content.

Assumptions & free parameters 5 free parameters · 6 assumptions · 0 invented entities

The performance claims rest on a standard simulated 6-DOF vehicle model, a Gauss-Markov current model, a T200 thruster model with delay and dead zone, a cubic RPM energy proxy, and hand-tuned reward coefficients. None of these is independently calibrated to physical data, and the domain randomization ranges are chosen by the authors without external validation.

free parameters (5)
  • Terminal reward weights (position, attitude) = 5, 3 (Eq. 34)
    Hand-tuned coefficients for the terminal exponential rewards; no sensitivity analysis is provided.
  • Process reward guidance coefficients = 4.5 (position), 2 (orientation)
    Hand-tuned weights for velocity-guidance reward terms in Eqs. (35)-(36).
  • Energy reward scaling = 12 and 2.5e-6 (Eq. 38)
    Exponential reward constants for the energy proxy; chosen by the authors.
  • Settling-time thresholds = 0.02 m, 2 deg
    Chosen thresholds define success rate and settling time in Section 5.1; the choice affects the reported settling time advantage.
  • DAgger history length L = 30 steps
    History buffer length for the adaptation module; architectural hyperparameter that affects estimation accuracy.
assumptions (6)
  • standard math 6-DOF rigid-body underwater vehicle dynamics with added mass, Coriolis, damping and restoring forces (Eq. 7)
    Taken from the standard model attributed to Da Silva et al.; assumed valid for the BlueROV2-like vehicle.
  • domain assumption Ocean current is irrotational, slowly varying, and evolves as a first-order Gauss-Markov process (Eq. 21)
    This model is used for training and evaluation; real currents may not follow this process, limiting sim-to-real transfer.
  • domain assumption T200 thruster dynamics with dead zone, saturation and 0.1 s delay are representative (Section 2.2, after Chu et al. 2025)
    The reported control performance depends on this actuator model.
  • domain assumption Propulsion energy is proportional to sum of (n_i/60)^3 (Eq. 37, after Pivano et al. 2008)
    Used both in the reward and the energy evaluation metric; approximate and only valid near low-speed station-keeping.
  • ad hoc to paper Domain randomization ranges in Table 2 cover the realistic model uncertainty
    No physical validation is provided; the ranges are chosen by the authors.
  • domain assumption Privileged latent features (static, dynamic, flow) can be accurately inferred from a 30-step history via a GRU
    Foundation of the distillation stage; performance is bounded by this learnability.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Knowledge-Distilled End-to-End Reinforcement Learning for Smooth 6-DOF Thrust Control and Rapid Adaptation to Ocean Currents in Remotely Operated Vehicles." pith.science (2026). https://pith.science/paper/VQATCP6C

@misc{pith2026260808598,
  author       = {Pith},
  title        = {Pith review of: Knowledge-Distilled End-to-End Reinforcement Learning for Smooth 6-DOF Thrust Control and Rapid Adaptation to Ocean Currents in Remotely Operated Vehicles},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VQATCP6C}},
  note         = {Machine review of arXiv:2608.08598}
}
read the original abstract

With the continuous improvement of computational capabilities, end-to-end reinforcement learning has been rapidly developed for remotely operated vehicles control. Nevertheless, existing end-to-end reinforcement-learningbased methods still face challenges in achieving optimal control under oceancurrent disturbances. In particular, there remains a lack of a unified control framework that can simultaneously achieve low steady-state tracking error, rapid transient response, energy-efficient operation, and smooth controlforce outputs under disturbances. To address the issue, this paper proposes the thrust smoothness rapid current adaptation proximal policy optimization (TSRCA-PPO) method which learns a near-optimal strategy by a twostage distillation learning framework. The core innovations of this work lie in the reward-function design and the privileged multi-encoder architecture. Ablation studies validate the effectiveness of each module. Simulation results demonstrate that the proposed TSRCA-PPO method consistently outperforms the conventional cascaded P-PID controller across all evaluation metrics. Specifically, TSRCA-PPO reduces the steady-state position error, steady-state attitude error, settling time, energy index, and thrustsmoothness index to 42.7%, 76.5%, 10.6%, 93.5%, and 15.9% of the corresponding P-PID values, respectively.

Figures

Figures reproduced from arXiv: 2608.08598 by the authors.

Figure 1
Figure 1. Coordinate transformation between the inertial frame and the body-fixed frame of the underwater vehicles. J1(Θ) =   cψcθ −sψcϕ + cψsθsϕ sψsϕ + cψcϕsθ sψcθ cψcϕ + sϕsθsψ −cψsϕ + sθsψcϕ −sθ cθsϕ cθcϕ   (5) J2(Θ) =   1 sϕtθ cϕtθ 0 cϕ −sϕ 0 sϕ/cθ cϕ/cθ   , θ ̸= π 2 (6) The vehicle dynamics model are given as follows(Da Silva et al., 2007): Mν˙ r + C(νr)νr + D(νr)νr + g(η) = τ c, (7) where νr is the velocity of t… view at source ↗
Figure 2
Figure 2. An example of a Gauss–Markov time-varying 3D ocean current over 30 s, with parameters Vc = 0.5494 m/s, α = 7.45◦ , and β = 132.93◦ . 11 [PITH_FULL_IMAGE:figures/full_fig_p011_2.png] view at source ↗
Figure 3
Figure 3. Overall framework of the two-stage training pipeline of TSRCA￾PPO. 4. Methodology 4.1. Overall Framework: Two-stage Training Pipeline of TSRCA-PPO The complete framework is shown in [PITH_FULL_IMAGE:figures/full_fig_p014_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Stage-I privileged PPO framework. The training implementation is built upon the CleanRL codebase (Huang et al., 2022). We use the Adam optimizer, and all network components in the proposed framework are implemented as lightweight multilayer perceptrons. All latent feat…
Figure 5
Figure 5. Figure 5: Stage-II adaptation module architecture. and convert quaternion error to degree errors as e ◦ q,t = 180 π 2 arccos(|⟨qt , qd ⟩|). (32) The terminal reward at the t-th time step is defined as: rterm,t = 5dp,t + 3dq,t. (33) where dp,t = e −20ep,t , dq,t = e −0.2e ◦ q,t .…
Figure 6
Figure 6. Figure 6: Terminal exponential reward terms for position and attitude errors. Similarly, the orientation guidance reward is defined as the signed scalar projection of the vehicle’s angular velocity onto the quaternion-error vector: rproc,q,t = 2⟨ωt ,sgn(qe,w,t) qe,v,t⟩. (36) Bot…
Figure 7
Figure 7. Figure 7: Stage-I training return curves for the proposed Privileged PPO architecture and seven ablation or baseline variants. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_7.png]
Figure 8
Figure 8. Figure 8: Architecture of the conventional cascaded P-PID controller using pseudoinverse thrust allocation. 5.2. Comparison with Conventional Methods [PITH_FULL_IMAGE:figures/full_fig_p026_8.png]
Figure 9
Figure 9. Figure 9: Comparison between TSRCA-PPO and P-PID under stepwise east–south–west–north ocean-current disturbances: position and attitude er￾rors, current input, and latent-feature estimation error. 28 [PITH_FULL_IMAGE:figures/full_fig_p028_9.png]
Figure 10
Figure 10. Figure 10: Individual-thruster force comparison between TSRCA-PPO and P-PID under stepwise east–south–west–north ocean-current disturbances. 29 [PITH_FULL_IMAGE:figures/full_fig_p029_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 31 canonical work pages

  1. [1]

    Journal of Marine Science and Engineering , volume=

    Gao, Xiaoyang and Li, Tieshan , title=. Journal of Marine Science and Engineering , volume=. 2024 , doi=

    work page 2024

  2. [2]

    Mathematics , volume=

    Deng, Liping and Tao, Jianguo , title=. Mathematics , volume=. 2025 , doi=

    work page 2025

  3. [3]

    IEEE Transactions on Robotics , volume=

    Online Approximate Optimal Station Keeping of a Marine Craft in the Presence of an Irrotational Current , author=. IEEE Transactions on Robotics , volume=. 2018 , doi=

    work page 2018

  4. [4]

    and Fossen, Thor I

    Johansen, Tor A. and Fossen, Thor I. , title=. Automatica , volume=. 2013 , doi=

    work page 2013

  5. [5]

    , title=

    Fossen, Thor I. , title=

    work page

  6. [6]

    OCEANS 2023 -- Limerick , pages=

    Propeller Characterization Testing of a Blue Robotics T200 Thruster , author=. OCEANS 2023 -- Limerick , pages=. 2023 , organization=

    work page 2023

  7. [7]

    IFAC-PapersOnLine , volume=

    Nonlinear MPC for combined motion control and thrust allocation of ships , author=. IFAC-PapersOnLine , volume=. 2020 , doi=

    work page 2020

  8. [8]

    Control Engineering Practice , volume=

    Reinforcement Learning-Based NMPC for Tracking Control of ASVs: Theory and Experiments , author=. Control Engineering Practice , volume=. 2022 , doi=

    work page 2022

Show all 34 references
  1. [9]

    Deep Reinforcement Learning Based Active Disturbance Rejection Control for

    Luo, Gaosheng and Zhang, Dong and Feng, Wei and Jiang, Zhe and Liu, Xingchen , journal=. Deep Reinforcement Learning Based Active Disturbance Rejection Control for. 2025 , doi=

    2025

  2. [10]

    Engineering Applications of Artificial Intelligence , volume=

    A General Motion Controller Based on Deep Reinforcement Learning for an Autonomous Underwater Vehicle with Unknown Disturbances , author=. Engineering Applications of Artificial Intelligence , volume=. 2023 , publisher=

    2023

  3. [11]

    IFAC-PapersOnLine , volume=

    Integrated Thrust Allocation and Learning-Enhanced MPC for ROV Control Under Model Uncertainties , author=. IFAC-PapersOnLine , volume=. 2025 , doi=

    2025

  4. [12]

    Ocean Engineering , volume=

    Dynamic Positioning Using Deep Reinforcement Learning , author=. Ocean Engineering , volume=. 2021 , doi=

    2021

  5. [13]

    arXiv preprint arXiv:2512.13359 , year=

    Fast Policy Learning for 6-DOF Position Control of Underwater Vehicles , author=. arXiv preprint arXiv:2512.13359 , year=

  6. [14]

    Robotics and Autonomous Systems , volume=

    Adaptive Low-Level Control of Autonomous Underwater Vehicles Using Deep Reinforcement Learning , author=. Robotics and Autonomous Systems , volume=. 2018 , publisher=

    2018

  7. [15]

    2025 IEEE International Conference on Robotics and Automation (ICRA) , pages=

    Learning to Swim: Reinforcement Learning for 6-DOF Control of Thruster-Driven Autonomous Underwater Vehicles , author=. 2025 IEEE International Conference on Robotics and Automation (ICRA) , pages=. 2025 , doi=

    2025

  8. [16]

    IEEE Robotics and Automation Letters , volume=

    Swim4Real: Deep Reinforcement Learning-Based Energy-Efficient and Agile 6-DOF Control for Underwater Vehicles , author=. IEEE Robotics and Automation Letters , volume=. 2025 , doi=

    2025

  9. [17]

    arXiv preprint arXiv:2503.09203 , year=

    MarineGym: A High-Performance Reinforcement Learning Platform for Underwater Robotics , author=. arXiv preprint arXiv:2503.09203 , year=

    arXiv

  10. [18]

    Proceedings of the Conference on Robot Learning , series=

    Learning by Cheating , author=. Proceedings of the Conference on Robot Learning , series=. 2020 , publisher=

    2020

  11. [19]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=

    On the Continuity of Rotation Representations in Neural Networks , author=. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages=. 2019 , month=

    2019

  12. [20]

    IEEE/CAA Journal of Automatica Sinica , volume=

    MPC-based motion planning and control enables smarter and safer autonomous marine vehicles: Perspectives and a tutorial survey , author=. IEEE/CAA Journal of Automatica Sinica , volume=. 2023 , doi=

    2023

  13. [21]

    arXiv preprint arXiv:2408.03539 , year=

    Deep Reinforcement Learning for Robotics: A Survey of Real-World Successes , author=. arXiv preprint arXiv:2408.03539 , year=

    arXiv

  14. [22]

    2025 , publisher=

    Song, Yunlong and Bian, Xue and Li, Ning and Li, Chao and Li, Fei and Tang, Chuanlong and Cheng, Guangliang and Xiang, Longxiang and Wang, Fei-Yue , booktitle=. 2025 , publisher=

    2025

  15. [23]

    2024 , eprint=

    Gradient-based Regularization for Action Smoothness in Robotic Control with Reinforcement Learning , author=. 2024 , eprint=

    2024

  16. [24]

    2021 IEEE International Conference on Robotics and Automation (ICRA) , pages=

    Regularizing Action Policies for Smooth Control with Reinforcement Learning , author=. 2021 IEEE International Conference on Robotics and Automation (ICRA) , pages=. 2021 , organization=

    2021

  17. [25]

    2021 , doi=

    Kumar, Ashish and Fu, Zipeng and Pathak, Deepak and Malik, Jitendra , booktitle=. 2021 , doi=

    2021

  18. [26]

    2024 , publisher=

    Xiao, Zhiyuan and Zhang, Xinyu and Zhou, Xiang and Zhang, Qingrui , booktitle=. 2024 , publisher=

    2024

  19. [27]

    Da Silva, J. E. and Terra, B. and Martins, R. and de Sousa, J. B. , booktitle=. Modeling and Simulation of the

  20. [28]

    Sensors , volume=

    Robust Position Control of an Over-actuated Underwater Vehicle under Model Uncertainties and Ocean Current Effects Using Dynamic Sliding Mode Surface and Optimal Allocation Control , author=. Sensors , volume=. 2021 , doi=

    2021

  21. [29]

    Journal of Marine Science and Engineering , volume=

    An Open-Source Benchmark Simulator: Control of a BlueROV2 Underwater Robot , author=. Journal of Marine Science and Engineering , volume=. 2022 , doi=

    2022

  22. [30]

    arXiv preprint arXiv:1707.06347 , year=

    Proximal policy optimization algorithms , author=. arXiv preprint arXiv:1707.06347 , year=

    arXiv

  23. [31]

    arXiv preprint arXiv:1506.02438 , year=

    High-Dimensional Continuous Control Using Generalized Advantage Estimation , author=. arXiv preprint arXiv:1506.02438 , year=

    arXiv

  24. [32]

    Journal of Machine Learning Research , volume=

    CleanRL: High-quality Single-file Implementations of Deep Reinforcement Learning Algorithms , author=. Journal of Machine Learning Research , volume=

  25. [33]

    Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , series=

    A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , author=. Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics , series=. 2011 , publisher=

    2011

  26. [34]

    IEEE Transactions on Control Systems Technology , volume=

    A four-quadrant thrust estimation scheme for marine propellers: Theory and experiments , author=. IEEE Transactions on Control Systems Technology , volume=. 2008 , publisher=

    2008

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.