Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

This paper claims that a policy trained entirely in simulation in under three minutes can control a real AUV's six-degree-of-freedom velocity and attitude with zero real-world tuning.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A DRL velocity controller for AUVs, trained in under 3 minutes in simulation, transfers zero-shot to a real underwater vehicle for agile 6DOF path-following.

T0 review reviewed 2026-08-03 challenge →

load-bearing objection Sim2Swim is a plausible and genuinely fast DRL velocity controller for AUVs, but its zero-shot claim is under-supported by purely qualitative pool trials; worth a serious referee but needs quantitative validation before I'd trust it. the 4 major comments →

arxiv 2512.08656 v2 pith:L5PMRKEH submitted 2025-12-09 cs.RO

Sim2Swim: Zero-Shot Velocity Control for Agile AUV Maneuvering in 3 Minutes

Lauritz Rismark Fosso , Herman Bi{\o}rn Amundsen , Marios Xanthidis , Sveinung Johan Ohrem This is my paper
classification cs.RO
keywords Underwater roboticsDeep reinforcement learningZero-shot sim2realVelocity control6DOF maneuveringDomain randomizationAUV controlIntegral action
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper is trying to establish that a deep reinforcement learning controller for a holonomic AUV can be trained entirely in simulation, in about three minutes on a laptop, and then deployed on the real vehicle with zero additional tuning. The controller tracks six-degree-of-freedom references: desired linear velocities in surge, sway, and heave, plus arbitrary orientation references. This is more demanding than station-keeping because both translation and rotation vary over time. The authors argue that if this works, marine operators can stop hand-tuning PID gains for every payload change and instead use one simulation-trained policy across configurations. They support the claim with pool experiments showing accurate velocity and attitude tracking, including with a 600-gram asymmetric ballast and extreme roll and pitch commands.

Core claim

Sim2Swim's central claim is that integral action appended to a learned velocity/attitude policy eliminates the steady-state errors seen in earlier DRL position controllers while preserving fast, robust convergence. The policy observes quaternion error, linear velocity error, angular velocity, and accumulated integrals of velocity and attitude error; outputs six normalized force/torque commands which are scaled by the vehicle's thrust-gain matrix; and is trained with domain randomization over mass, volume, and the offset between center of buoyancy and center of mass. In pool trials on an instrumented ROV, the same zero-shot policy tracks straight-line paths, a square path under an added 600 g

What carries the argument

A two-layer MLP policy trained with proximal policy optimization in thousands of parallel, short (5-second) simulated episodes. Its observation vector combines quaternion error, linear velocity error, angular velocity, and integral states of velocity and attitude error; the integral states give the policy memory and remove steady-state error. Actions are six normalized body-frame force/torque commands mapped through the thrust-gain matrix K, so the policy need not learn thruster allocation. Domain randomization perturbs mass, volume, and the center-of-buoyancy offset relative to the center of mass to force transferable behavior.

Load-bearing premise

The load-bearing premise is that a rigid-body simulator with randomized mass, volume, and buoyancy offset, and 5-second episodes, captures enough of the real vehicle's hydrodynamics and disturbances that a policy trained only in simulation transfers to the pool without any real-world adjustment.

What would settle it

Deploy the trained policy on the same ROV while a controlled current or wave disturbance is applied and measure whether velocity and quaternion errors return to zero once references stabilize; if they remain biased or oscillate under a disturbance within the vehicle's rated operating envelope, the zero-shot claim fails. A second, simpler test is to apply the policy to a different ROV with a different thruster layout without retraining—if tracking degrades, the controller is not platform-general.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Controller retuning between deployments becomes unnecessary: a single policy trained once in simulation carries over across payloads within the randomized range.
  • Attitude can be commanded independently of path, enabling inspection maneuvers where the vehicle holds a fixed orientation to a structure while translating along it.
  • Integral observations close the steady-state velocity and pitch/sway errors that earlier learning-based AUV controllers left behind.
  • Reducing training from hours to minutes on a laptop makes DRL-based control a realistic option for small field robots without specialized infrastructure.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the recipe generalizes, any holonomic vehicle with six actuated DOFs — not just this ROV — could inherit the same zero-shot pipeline, needing only a randomized rigid-body model.
  • The decisive next test is environmental disturbance: currents and waves are absent from the 5-second training episodes, so the claim of generality beyond calm pools is unproven.
  • The authors' 3-minute budget suggests sample efficiency is no longer the bottleneck; simulator fidelity and the coverage of domain randomization are what determine where zero-shot transfer ends.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents Sim2Swim, a reinforcement-learning-based controller for holonomic AUVs that outputs desired forces/torques for velocity and attitude tracking. Trained in Isaac Lab with PPO and domain randomization, the policy uses integral observations of velocity and orientation errors to reduce steady-state error. The authors claim zero-shot sim2real transfer after only 3 minutes of training, and validate with three pool trials on a BlueROV2 Heavy: straight-line path following, a ballast perturbation, and a square path with random attitude setpoints. The central claim is that the trained policy generalizes across payloads and can perform agile 6DOF maneuvering without any real-world fine-tuning.

Significance. If supported, this would be a meaningful step toward practical DRL for AUV control: fast training, no per-platform retuning, and demonstrated agility in attitude and velocity tracking. The proposed integral-observation mechanism is a plausible approach to remove steady-state error observed in prior work, and the choice of a force/torque action space rather than thruster-level commands is a sensible design decision that maintains platform generality. Hardware validation in three configurations is a useful contribution, provided the performance is quantified. The paper also brings together recent work on massively parallelized RL and AUV sim2real.

major comments (4)
  1. [§5.1, Figure 3, Eq. (5)–(8)] The reported mean reward of 315 in the final learning iteration is inconsistent with the reward definition. Per step, each term in Eq. (5) is no greater than its weight; with Table 1 weights (0.4+0.05+0.2+0.3), the maximum per-step reward is 0.95. With a 5 s episode at 10 Hz, the maximum episode reward is 47.5, not 315. Unless there is an omitted scaling, discounting, or a different reward normalization, Figure 3 contradicts Eq. (5). This undermines the '3-minute convergence' evidence and must be resolved before the training claim is credible.
  2. [§5.2, Figure 4] The zero-shot claim rests on qualitative descriptions of three single pool trials. No tracking-error metrics (e.g., RMS or MAE of velocity and attitude errors), no repeated trials, and no confidence intervals are reported. The text itself admits nontrivial deviations: 'unable to track fast variations' in sway and heave, 'slower response in heave', a depth deviation from the desired path, and attitude offsets attributed to thrust allocation. Without quantitative performance measures, the reader cannot distinguish robust zero-shot control from marginal or trial-selected behavior. This is load-bearing for the paper's central contribution and should be addressed with quantitative error metrics and at least repeated runs.
  3. [§4.4, §5.2 (ballast trial)] Domain randomization ranges for mass, volume, and CB–CM offset are not specified. The paper states only that mass and volume are 'uniformly sampled' and the CB–CM offset is 'uniformly sampled in a sphere'. To assess whether the real ballast perturbation (600 g, ~5% mass change, buoyancy sign change) is covered by the randomization, the ranges are essential. Without them, the transferability argument is not reproducible and the zero-shot claim is under-supported. Please provide the exact distributions and justify their coverage of the real parametric variation.
  4. [§5.2, §6] The paper advocates against hand-tuned PID controllers but provides no baseline comparison to a tuned PID or to prior DRL controllers (e.g., Cai et al. 2025 or Sufán and Troni 2025). The claim that Sim2Swim shows 'superior performance' or 'robust control' is not substantiated without a comparative baseline under the same test conditions. Adding a baseline, even for a subset of trials, would place the results in context and support the 'general' claim.
minor comments (6)
  1. [§4.2] The sentence describing the policy architecture is incomplete: 'The policy is realized as a 2-layer MLP with, and is trained...'. Please specify the hidden dimension and activation function.
  2. [Table 1] The row 'Actions w_i 0.3' presumably refers to the action reward weight w_a, not the integral-state weight. Please correct the notation.
  3. [Eq. (5)–(8)] The notation r_i and w_i is ambiguous: Eq. (6) defines r_i generically, but the terms for orientation, angular velocity, and linear velocity all use different weights. It would be clearer to write r_omega, r_v, and separate r_q or explain how the three r_i terms are combined.
  4. [Figure 4] The attitude subplot labels appear garbled (e.g., '3A', '?d3dAd'). The roll/pitch/yaw curves should be clearly identified. Also, the time axes are not aligned across rows, making comparison difficult.
  5. [§4.5] The desired speed is fixed at V_d = 0.5 m/s during training. The pool trials appear to use similar speeds, but a statement on how well the policy performs outside this training speed would strengthen the generality claim.
  6. [§5.2] Please state whether the three trials are single runs or representative of multiple runs, and report the number of trial repetitions, if any.

Circularity Check

0 steps flagged

No material circularity: the policy is empirically trained and independently validated; minor self-citation is not load-bearing.

full rationale

The paper does not derive Sim2Swim from its own target claim. The policy is trained with PPO (Schulman et al. 2017; Rudin et al. 2022) in Isaac Lab on a randomized rigid-body AUV model, and the zero-shot claim is tested against independent physical pool trials on a BlueROV2 with added ballast and random orientation setpoints. The reward weights, domain randomization ranges, and episode length are hand-chosen inputs, but they are not fitted to the pool results nor renamed as predictions; similarly, the LOS guidance law is an external standard method. The only overlapping-author reference (Ohrem et al. 2024) appears as background on adaptive control and does not support any load-bearing inference. The paper itself reports unquantified tracking deviations in sway, heave, and attitude in Section 5.2, and there are internal-reporting concerns such as the apparent inconsistency between Figure 3's mean reward of 315 and the bounded reward in Eq. 5; these are correctness/validation weaknesses, not circularity. No equation or claim is shown to equal its own input by construction.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The central claim rests on the unproven assumption that a policy trained in a randomized rigid-body simulation transfers to a real AUV without fine-tuning. The main free parameters are reward weights, training trajectory coefficients, and domain randomization ranges; none are derived from first principles.

free parameters (5)
  • reward weights w_q, w_omega, w_v, w_i = 0.4, 0.05, 0.2, 0.3
    Hand-chosen in Table 1; no tuning procedure or sensitivity analysis, directly shape policy behavior.
  • trajectory coefficients [a,b,c] and frequency omega = [0.5, 0.5, 0.3], 0.2
    Hand-chosen desired-state distribution for training in §4.5; affects the range of maneuvers the policy learns.
  • desired training speed V_d = 0.5 m/s
    Fixed in §4.5; no justification for this speed.
  • domain randomization ranges for mass, volume, CB-CM offset = not specified
    Mentioned in §4.4 but ranges are not reported, so the robustness claim cannot be assessed.
  • episode length and number of environments = 5 s, 2048
    Chosen in §5.1; affect convergence and transfer, no ablation.
axioms (4)
  • domain assumption The AUV dynamics are governed by the standard 6-DOF rigid-body equations from Fossen (2021), Eq. (1).
    The simulation and real vehicle are assumed to follow this model; any significant unmodeled dynamics would break the sim-to-real transfer.
  • domain assumption Domain randomization of mass, volume, and CB-CM offset in simulation produces a policy robust to real parametric uncertainties.
    The paper asserts this in §4.4 without providing the randomization ranges or an ablation to demonstrate that randomization (rather than the nominal sim model) is what transfers.
  • domain assumption The policy trained on short 5-second episodes with random waypoint trajectories generalizes to longer real-world paths.
    Training episodes are limited to 5 s (§5.1) yet the pool trials last 30–50 s with multiple waypoints; no evidence is given that the policy extrapolates in time.
  • ad hoc to paper Integral observations of velocity and orientation errors, despite not being rewarded, provide sufficient memory for the policy to achieve zero steady-state error.
    The integral states are added to the observation (Eq. 3) with no reward terms (§4.3); the claim of zero steady-state error relies on this design working as intended.

reviewed 2026-08-03 · how reviews work

0 comments
Cite this review

Pith. "Pith review of Sim2Swim: Zero-Shot Velocity Control for Agile AUV Maneuvering in 3 Minutes." pith.science (2026). https://pith.science/paper/L5PMRKEH

@misc{pith2026251208656,
  author       = {Pith},
  title        = {Pith review of: Sim2Swim: Zero-Shot Velocity Control for Agile AUV Maneuvering in 3 Minutes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/L5PMRKEH}},
  note         = {Machine review of arXiv:2512.08656}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Holonomic autonomous underwater vehicles (AUVs) have the hardware ability for agile maneuvering in both translational and rotational degrees of freedom (DOFs). However, due to challenges inherent to underwater vehicles, such as complex hydrostatics and hydrodynamics, parametric uncertainties, and frequent changes in dynamics due to payload changes, control is challenging. Performance typically relies on carefully tuned controllers targeting unique platform configurations, and a need for re-tuning for deployment under varying payloads and hydrodynamic conditions. As a consequence, agile maneuvering with simultaneous tracking of time-varying references in both translational and rotational DOFs is rarely utilized in practice. To the best of our knowledge, this paper presents the first general zero-shot sim2real deep reinforcement learning-based (DRL) velocity controller enabling path following and agile 6DOF maneuvering with a training duration of just 3 minutes. Sim2Swim, the proposed approach, inspired by state-of-the-art DRL-based position control, leverages domain randomization and massively parallelized training to converge to field-deployable control policies for AUVs of variable characteristics without post-processing or tuning. Sim2Swim is extensively validated in pool trials for a variety of configurations, showcasing robust control for highly agile motions.

Figures

Figures reproduced from arXiv: 2512.08656 by Herman Bi{\o}rn Amundsen, Lauritz Rismark Fosso, Marios Xanthidis, Sveinung Johan Ohrem.

Figure 1
Figure 1. Figure 1: An instance of massively parallelized training with [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Sim2Swim: The proposed method receives desired [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Mean reward against training time. 5.2 Sim2Real Transfer We validate the policy on a BlueRobotics BlueROV2 Heavy, depicted in Figures 1b and 1c in an indoor pool. The vehicle is equipped with a Water Linked A50 Doppler velocity log measuring body velocity v b and estimating position (x and y). We measure the depth (z, positive down) with a BlueRobotics Bar30 pressure sensor, while the orientation is provid… view at source ↗
Figure 4
Figure 4. Figure 4: Experiment results of three separate trials. Left column shows the ROV following a straight line, where the desired [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. CORAL-AUV: CFD Oriented Reinforcement Learning for Autonomous Underwater Vehicles

    cs.RO 2026-07 conditional novelty 6.5

    Surrogate CFD drag models enable zero-shot RL control of a 6-DOF AUV that is more efficient, accurate, and robust to payload change than simplified-physics baselines.

Reference graph

Works this paper leans on

6 extracted references · 1 linked inside Pith · cited by 1 Pith paper

  1. [1]

    Bingham, B., Foley, B., Singh, H., Camilli, R., Delaporta, K., Eustice, R., Mallios, A., Mindell, D., Roman, C., and Sakellariou, D. (2010). Robotic tools for deep water archaeology: Surveying an ancient shipwreck with an autonomous underwater vehicle.Journal of Field Robotics, 27(6), 702–717. Breivik, M. and Fossen, T. (2005). Principles of guidance- bas...

    2010

  2. [20]

    Xanthidis, M., Kalaitzakis, M., Karapetyan, N., Johnson, J., Vitzilaios, N., O’Kane, J.M., and Rekleitis, I. (2021). AquaVis: A Perception-Aware Autonomous Navigation Framework for Underwater Vehicles. In2021 IEEE/RSJ International Conference on Intelligent Robots and Sys- tems (IROS), volume 1, 5410–5417

    2021

  3. [86]

    Diamanti, E., Fossdal, M., Iversflaten, M.H., Sæbø, B.K., Kasparaviˇ ci¯ ut˙ e, G., Waldum, A.G., Yip, M., Ødeg ˚ ard, Ø., De La Torre, P., Pettersen, K.Y., et al. (2025). Marine archaeological surveying using snake robots: The eely survey of figaro wreck in the high arctic.Marine Technology Society Journal, 59(2), 78–103. Eschmann, J., Albani, D., and Lo...

    2025

  4. [769]

    and Troni, G

    Suf´ an, V. and Troni, G. (2025). Swim4Real: Deep Re- inforcement Learning-Based Energy-Efficient and Agile 6-DOF Control for Underwater Vehicles.IEEE Robotics and Automation Letters, 10(7), 7326–7333. Teigland, H., Hassani, V., and Møller, M.T. (2020). Op- erator focused automation of ROV operations. In2020 IEEE/OES Autonomous Underwater Vehicles Sympo- ...

    2025

  5. [866]

    Rudin, N., Hoeller, D., Reist, P., and Hutter, M. (2022). Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning. ArXiv:2109.11978 [cs]. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., and Klimov, O. (2017). Proximal Policy Optimization Algo- rithms. ArXiv:1707.06347 [cs]. Shtessel, Y., Taleb, M., and Plestan, F. (2012). A n...

    Pith/arXiv arXiv 2022

  6. [3706]

    Wang, J., Xiang, S., Shen, T., Fang, Z., Niu, S., Pan, X., and Li, G. (2025). Imitation learning from observation for ROV path tracking.Intelligent Marine Technology and Systems, 3(1),

    2025

This paper was first reviewed by deepseek-v4-flash on August 3, 2026.