Pith. sign in

REVIEW 3 major objections 4 minor 30 references

Surrogate CFD drag models let RL controllers for 6-DOF AUVs transfer zero-shot with lower energy, faster waypoints, and payload robustness that simpler physics models lack.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-13 02:09 UTC pith:WV57H7KU

load-bearing objection First real zero-shot 6-DOF AUV RL from CFD surrogates; payload transfer is clean evidence, field percentages partly confounded by reward choice. the 3 major comments →

arxiv 2607.09557 v1 pith:WV57H7KU submitted 2026-07-10 cs.RO

CORAL-AUV: CFD Oriented Reinforcement Learning for Autonomous Underwater Vehicles

Steven Roche , Milo Van Mooy , Nathan McGuire , Levi Cai , Jonathan P. How , Yogesh Girdhar This is my paper
classification cs.RO
keywords sim-to-real transferreinforcement learningautonomous underwater vehiclescomputational fluid dynamicssurrogate drag modelsdomain randomization6-DOF control
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Fine control of autonomous underwater vehicles is hard because traditional controllers need retuning whenever payload or conditions change, and simplified drag models leave a large sim-to-real gap. This paper shows that lightweight neural networks trained on steady-state CFD data can stand in for full fluid-dynamics solvers inside a reinforcement-learning loop, producing a controller in minutes. The resulting policy is the first zero-shot RL controller for a full 6-DOF AUV trained this way. In field and tank tests it uses 31 percent less thruster energy, reaches waypoints 11 percent faster with 19 percent less path error, and is the only policy that still works after two pounds are added to the stern. The higher-fidelity drag model also makes training less sensitive to reward-shaping choices and more predictive of real-world transfer.

Core claim

Training an RL policy on surrogate drag models that approximate steady-state CFD data yields a zero-shot 6-DOF AUV controller that outperforms both an equivalent-inertia-box model and a system-identified diagonal model: lower energy, faster transit, smaller tracking error, greater robustness to reward design, and successful transfer under a 2 lb stern payload that defeats the other two policies.

What carries the argument

Surrogate drag models (SDMs): two small MLPs, one for linear velocity and one for angular velocity, each fitted to large RANS CFD wrench datasets and summed at every simulation step so that high-fidelity coupled drag can be queried in real time inside a massively parallel RL environment.

Load-bearing premise

That sampling linear and angular velocities independently in steady-state CFD and then simply adding the two MLP outputs is faithful enough to the vehicle’s real coupled, possibly unsteady hydrodynamics for zero-shot transfer to succeed.

What would settle it

Repeat the identical tank and field waypoint trials with a new payload or vehicle geometry whose true hydrodynamics deviate strongly from the steady-state RANS assumption; if the SDM policy then fails while a full transient CFD or real-data model succeeds, the central claim is falsified.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes training lightweight MLP surrogate drag models (SDMs) on steady-state OpenFOAM RANS data of a 6-DOF AUV, then using those SDMs inside a highly parallel IsaacSim PPO loop so that zero-shot policies can be obtained in ~15 minutes. Three drag models are compared (MuJoCo-style inertia box, System-ID diagonal coefficients, and the CFD SDM). On a field U-pattern the SDM policy reports 31 % lower thruster effort, 11 % faster waypoint times and 19 % lower cross-track error than the simplified-physics baseline; in a tank payload experiment only the SDM policy transfers under a 2 lb stern weight when domain randomization is applied. The authors further claim that the CFD-based controller is less sensitive to reward-coefficient choices and that its simulation reward curves better predict real-world transfer.

Significance. If the causal attribution to hydrodynamic fidelity holds, the work supplies a practical route for incorporating high-fidelity CFD into RL pipelines without placing a full solver inside the training loop—an important engineering contribution for 6-DOF AUV control. The combination of tank, field (including a scientifically relevant reef site) and payload-transfer experiments is stronger empirical grounding than most contemporaneous AUV-RL papers. Explicit credit is due for the open comparison across three drag models, the quantitative energy/time/error metrics, and the demonstration that only the CFD policy survives a realistic payload change under domain randomization.

major comments (3)
  1. §4.1 / Figs. 3–4: the headline 31 % / 11 % / 19 % gains are obtained under a single hand-chosen reward set (§3.3.1) with n=5 trials per policy. The tank reward-shaping sweep (§3.3.2, Fig. 5, A.1) already shows that the inertia-box model can be made competitive simply by switching from “Aggressive” to “Balanced” coefficients. Without a matched multi-reward field ablation (or at least confidence intervals / statistical tests on the n=5 runs), the causal claim that the gains arise from CFD fidelity rather than differential brittleness to that particular reward remains under-supported.
  2. §3.1.3: the SDM is constructed by sampling linear and angular velocities independently, fitting two separate MLPs, and summing their wrench outputs. The manuscript never quantifies the approximation error of this split relative to a joint (v,ω) CFD evaluation or to measured vehicle wrenches. Because the central scientific claim is that higher-fidelity hydrodynamics close the sim-to-real gap, an error analysis (or at least a residual plot against held-out joint-velocity CFD cases) is load-bearing and currently missing.
  3. §4.2 / Fig. 6: the exclusive 2 lb transfer success is the strongest evidence for the method, yet it still uses only the single best reward found for each model and five trials. Reporting success rates with binomial confidence intervals (or repeating the payload test under the other two reward sets) would make the robustness claim far more convincing.
minor comments (4)
  1. Abstract and §1 claim “first” zero-shot 6-DOF AUV policy trained on CFD surrogates; a short related-work paragraph clarifying the distinction from the 2-D/3-DOF CFD-RL fish papers already cited would strengthen the novelty statement.
  2. Eq. (1) and the System-ID paragraph: the effective-mass / inertia symbols are never numerically reported; listing the fitted d_i, k_j values (or placing them in the appendix) would aid reproducibility.
  3. Figure A.2 caption: the red inertia-box curves appear only for a subset of axes; clarifying which components are shown would help the reader assess the claimed cross-coupling.
  4. Typographical: “PIMPLE Solver” appears twice in Fig. 1 with inconsistent capitalization; “meff,i” in Eq. (1) should be m_eff,i for consistency with later notation.

Circularity Check

0 steps flagged

No circularity: zero-shot performance claims are measured on independent real hardware after training on CFD-derived surrogates and simulation; nothing reduces by construction to its inputs.

full rationale

The derivation chain is: generate independent steady-state CFD wrenches (OpenFOAM RANS), fit lightweight MLP surrogates, insert them into an IsaacSim PPO loop (with optional DR), then deploy the resulting policy zero-shot onto the physical 6-DOF AUV and measure energy, time, cross-track error, and waypoint success on tank and field trajectories never seen during training. The 31%/11%/19% field numbers (Figs. 3-4) use a single fixed reward recipe stated a priori in §3.3.1; the System-ID baseline is fitted only to coast-down data and is used solely as a comparator, not as a source of the claimed gains. Reward-coefficient iteration in §3.3.2 is ordinary hyper-parameter search on tank data and does not redefine the field or 2 lb transfer metrics. Self-citations to the authors’ prior pipeline (Cai et al.) supply the base RL infrastructure but are not load-bearing uniqueness theorems or ansatzes that force the CFD-SDM results. No equation equates a fitted quantity to a “prediction” of itself, and no central claim collapses by definition. The paper is therefore free of the circular patterns enumerated in the analyzer guidelines.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 1 invented entities

The central claim rests on standard fluid-dynamics and RL machinery plus a handful of modeling choices (steady-state RANS, linear/angular split, MLP capacity, DR ranges, reward weights) that are either fitted or selected by the authors. No new physical entities are postulated; the free parameters are the usual engineering knobs of a sim-to-real pipeline.

free parameters (4)
  • reward coefficients (λ_p, λ_q, λ_stable, λ_v, λ_w, λ_a)
    Hand-chosen and iteratively adjusted after tank tests; three different sets are reported and the best retained for each drag model.
  • domain-randomization ranges (COB-COM offset 0.01–0.02 m, volume ±1–2 %)
    Chosen to match expected 1–2 lb payload shifts; not derived from first principles.
  • MLP architecture (3×1024 units) and training split
    Selected on a validation set of the CFD data; capacity and regularization are free design choices.
  • System-ID drag coefficients d_i, k_j
    Fitted by least-squares to coast-down trials; used as a baseline dynamics model.
axioms (3)
  • domain assumption Steady-state RANS (k-ω SST) CFD with independent linear and angular velocity sampling adequately captures the dominant hydrodynamic forces for control purposes.
    Section 3.1.3; the entire SDM pipeline rests on this modeling choice.
  • ad hoc to paper Summing separate linear-velocity and angular-velocity MLP outputs yields a usable total wrench.
    Explicit design decision to reduce CFD sampling cost from O(n^6) to two lower-dimensional datasets.
  • domain assumption PPO with 10 000 parallel IsaacSim environments converges to a transferable policy under the stated observation and action spaces.
    Standard RL practice; success is demonstrated empirically rather than proved.
invented entities (1)
  • Surrogate Drag Model (SDM) no independent evidence
    purpose: Fast neural approximation of CFD wrench that can be queried inside the RL training loop.
    Defined as the pair of MLPs trained on the OpenFOAM datasets; no independent physical existence claimed beyond the approximation task.

pith-pipeline@v1.1.0-grok45 · 15994 in / 2788 out tokens · 30209 ms · 2026-07-13T02:09:07.254099+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of CORAL-AUV: CFD Oriented Reinforcement Learning for Autonomous Underwater Vehicles." pith.science (2026). https://pith.science/paper/WV57H7KU

@misc{pith2026260709557,
  author       = {Pith},
  title        = {Pith review of: CORAL-AUV: CFD Oriented Reinforcement Learning for Autonomous Underwater Vehicles},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WV57H7KU}},
  note         = {Machine review of arXiv:2607.09557}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Fine grain control and positioning of autonomous underwater vehicles (AUVs) is critical for sampling, maintenance, and survey applications. Traditional control methods for AUVs are labor intensive and are not robust to changes in the vehicle configuration or environmental conditions. Reinforcement learning (RL) promises rapid controller development while handling a range of deployment parameters via domain randomization (DR). However, DR is still limited by the capacity of the underlying simulation to model real physics. In particular, drag physics are difficult to model and are a large contributor to sim-to-real gaps. Meanwhile, computational fluid dynamics (CFD) provides high fidelity drag models but is challenging to leverage within reinforcement learning frameworks due to its computational overhead. Thus, in this paper we exploit the idea of training surrogate approximations of CFD models of a given vehicle, enabling fast inference within RL pipelines. We are the first to successfully deploy a zero-shot RL policy on a 6-DOF AUV in which policy training is performed on surrogate drag models (SDMs) trained on CFD data. We find 31% lower energy usage compared to a controller using simplified physics while traversing between waypoints 11% faster with 19% less error. Our SDM based RL controller better predicts zero-shot transfer and is more robust across reward shaping design choices. When using DR to complete a task with perturbed parameters, we find that the CFD policy is the only controller that successfully transfers. The policies are evaluated in a controlled tank environment and in the field providing extensive testing of the policies' capabilities.

Figures

Figures reproduced from arXiv: 2607.09557 by Jonathan P. How, Levi Cai, Milo van Mooy, Nathan McGuire, Steven Roche, Yogesh Girdhar.

Figure 2
Figure 2. Figure 2: Example of OpenFOAM Steady State CFD flow calculation. For system ID, we assume a linear diagonal drag model, in which each dominant translational or ro￾tational axis is modeled independently, neglecting cross-axis coupling. We estimate the drag coeffi￾cients from coast-down data collected on the real ve￾hicle using IMU and DVL measurements fused with an Extended Kalman Filter. A coast-down trial con￾sists… view at source ↗
Figure 1
Figure 1. Figure 1: Overview of our method. We incorporate the surrogate models directly into IsaacSim, [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: On a U-Shape pattern in an uncontrolled environment, we find on a per segment basis [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: On a U-Shape pattern in an uncontrolled environment, we find on a per-episode basis that [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparing the best policies discovered during the reward shaping experiment, we find [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: When we place 2 lbs on the vehicle, only the CFD based policy transfers successfully [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

30 extracted references · 6 canonical work pages

  1. [1]

    J. Das, F. Py, J. B. Harvey, J. P. Ryan, A. Gellene, R. Graham, D. A. Caron, K. Rajan, and G. S. Sukhatme. Data-driven robotic sampling for marine ecosystem monitoring.International Journal of Robotics Research, 2015

    2015

  2. [2]

    L. Cai, N. E. McGuire, R. Hanlon, T. A. Mooney, and Y . Girdhar. Semi-supervised visual tracking of marine animals using autonomous underwater vehicles.International Journal of Computer Vision, 2023

    2023

  3. [3]

    D. R. Yoerger, A. F. Govindarajan, J. C. Howland, J. K. Llopiz, P. H. Wiebe, M. Curran, J. Fujii, D. Gomez-Ibanez, K. Katija, B. H. Robison, B. W. Hobson, M. Risi, and S. M. Rock. A hybrid underwater robot for multidisciplinary investigation of the ocean twilight zone.Science Robotics, 2021

    2021

  4. [4]

    Manderson, J

    T. Manderson, J. C. G. Higuera, S. Wapnick, J. F. Tremblay, F. Shkurti, D. Meger, and G. Dudek. Vision-based goal-conditioned policies for underwater navigation in the presence of obstacles. InRobotics: Science and Systems, 2020

    2020

  5. [5]

    Vivekanandan, D

    R. Vivekanandan, D. Chang, and G. A. Hollinger. Autonomous underwater docking using flow state estimation and model predictive control. InProceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2023

    2023

  6. [6]

    Xanthidis et al

    M. Xanthidis et al. Navigation in the presence of obstacles for an agile autonomous underwater vehicle. InProceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2020

    2020

  7. [7]

    Girdhar, N

    Y . Girdhar, N. McGuire, L. Cai, S. Jamieson, S. McCammon, B. Claus, J. E. S. Soucie, J. E. Todd, and T. A. Mooney. Curee: A curious underwater robot for ecosystem exploration. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2023

    2023

  8. [8]

    C. Li, S. Guo, and J. Guo. Study on obstacle avoidance strategy using multiple ultrasonic sensors for spherical underwater robots.IEEE Sensors Journal, 22(24), 2022

    2022

  9. [9]

    L. A. Hawkes, O. Exeter, S. M. Henderson, et al. Autonomous underwater videography and tracking of basking sharks.Animal Biotelemetry, 8(29), 2020

    2020

  10. [10]

    Zhu, J.-H

    Y . Zhu, J.-H. Pang, and F.-B. Tian. Point-to-point navigation of a fish-like swimmer in a vortical flow with deep reinforcement learning.Frontiers in Physics, V olume 10 - 2022, 2022. ISSN 2296-424X. doi:10.3389/fphy.2022.870273. URLhttps://www.frontiersin.org/ journals/physics/articles/10.3389/fphy.2022.870273

    work page doi:10.3389/fphy.2022.870273 2022

  11. [11]

    A. K. Lidtke, D. Rijpkema, and B. D ¨uz. General reinforcement learning control for auv manoeuvring in turbulent flows.Ocean Engineering, 309:118538, 2024. ISSN 0029-8018. doi:https://doi.org/10.1016/j.oceaneng.2024.118538. URLhttps://www.sciencedirect. com/science/article/pii/S0029801824018766

    work page doi:10.1016/j.oceaneng.2024.118538 2024

  12. [12]

    X. Cui, B. Sun, Y . Zhu, N. Yang, H. Zhang, W. Cui, D. Fan, and J. Wang. Enhancing efficiency and propulsion in bio-mimetic robotic fish through end-to-end deep reinforcement learning. Physics of Fluids, 36(3), Mar. 2024. ISSN 1089-7666. doi:10.1063/5.0192993. URLhttp: //dx.doi.org/10.1063/5.0192993. 9

    work page doi:10.1063/5.0192993 2024

  13. [13]

    Zhang, R

    T. Zhang, R. Tian, H. Yang, C. Wang, J. Sun, S. Zhang, and G. Xie. From simulation to reality: A learning framework for fish-like robots to perform control tasks.IEEE Transactions on Robotics, PP:1–18, 12 2022. doi:10.1109/TRO.2022.3181014

    doi:10.1109/tro.2022.3181014 2022

  14. [14]

    X. Lin, X. Liu, and Y . Wang. Learning agile swimming: An end-to-end approach without cpgs,

  15. [15]

    URLhttps://arxiv.org/abs/2409.10019

    Pith/arXiv arXiv

  16. [16]

    L. Cai, K. Chang, and Y . Girdhar. Learning to swim: Reinforcement learning for 6-dof control of thruster-driven autonomous underwater vehicles. InProceedings of the IEEE International Conference on Robotics and Automation (ICRA), 2025

    2025

  17. [17]

    Sufn and G

    V . Sufn and G. Troni. Swim4Real: Deep Reinforcement Learning-Based Energy-Efficient and Agile 6-DOF Control for Underwater Vehicles.IEEE Robotics and Automation Letters, 10 (7):7326–7333, July 2025. ISSN 2377-3766. doi:10.1109/LRA.2025.3575650. URLhttps: //ieeexplore.ieee.org/document/11020757/

    work page doi:10.1109/lra.2025.3575650 2025

  18. [18]

    Tunay, A

    S. Tunay, A. Andres, and I. Carlucho. Fast policy learning for 6-dof position control of under- water vehicles, 2026. URLhttps://arxiv.org/abs/2512.13359

    arXiv 2026

  19. [19]

    L. R. Fosso, H. B. Amundsen, M. Xanthidis, and S. J. Ohrem. Sim2swim: Zero-shot veloc- ity control for agile auv maneuvering in 3 minutes, 2025. URLhttps://arxiv.org/abs/ 2512.08656

    Pith/arXiv arXiv 2025

  20. [20]

    Y . Weng, S. Chun, M. Ohashi, T. Matsuda, Y . Sekimori, J. Pajarinen, J. Peters, and T. Maki. Autonomous underwater vehicle link alignment control in unknown environments using re- inforcement learning.Journal of Field Robotics, 41(6):1724–1743, 2024. ISSN 1556-4967. doi:10.1002/rob.22348. URLhttps://onlinelibrary.wiley.com/doi/abs/10.1002/ rob.22348. epr...

    work page doi:10.1002/rob.22348 2024

  21. [21]

    T. Liu, Y . Hu, and H. Xu. Deep Reinforcement Learning for Vectored Thruster Autonomous Underwater Vehicle Control.Complexity, 2021(1):6649625, 2021. ISSN 1099-0526. doi: 10.1155/2021/6649625. URLhttps://onlinelibrary.wiley.com/doi/abs/10.1155/ 2021/6649625. eprint: https://onlinelibrary.wiley.com/doi/pdf/10.1155/2021/6649625

    work page doi:10.1155/2021/6649625 2021

  22. [22]

    S. Bhat, C. Panteli, I. Stenius, and D. V . Dimarogonas. Nonlinear model predictive control for hydrobatics: Experiments with an underactuated auv.Journal of Field Robotics, 2023

    2023

  23. [23]

    Mitchell, E

    A. Mitchell, E. McGookin, and D. Murray-Smith. Comparison of control methods for au- tonomous underwater vehicles. InIF AC Workshop on Guidance and Control of Underwa- ter V ehicles, volume 36, pages 37–42, Newport, South Wales, UK, 2003. URLhttps: //www.sciencedirect.com/science/article/pii/S1474667017366545

    2003

  24. [24]

    W. Liu, K. Bai, X. He, S. Song, C. Zheng, and X. Liu. Fishgym: A high-performance physics- based simulation framework for underwater robot learning, 2022. URLhttps://arxiv. org/abs/2206.01683

    Pith/arXiv arXiv 2022

  25. [25]

    Todorov, T

    E. Todorov, T. Erez, and Y . Tassa. Mujoco: A physics engine for model-based control.2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 5026–5033,

    2012

  26. [26]

    URLhttps://api.semanticscholar.org/CorpusID:5230692

  27. [27]

    URLhttps://www.openfoam.com/

    OpenFOAM, June 2025. URLhttps://www.openfoam.com/

    2025

  28. [28]

    H. G. Weller, G. Tabor, H. Jasak, and C. Fureby. A tensorial approach to computational con- tinuum mechanics using object-oriented techniques.Computers in Physics, 12(6), 1998

    1998

  29. [29]

    Molchanov, T

    A. Molchanov, T. Chen, W. Hnig, J. A. Preiss, N. Ayanian, and G. S. Sukhatme. Sim-to- (multi)-real: Transfer of low-level robust control policies to multiple quadrotors, 2019. URL https://arxiv.org/abs/1903.04628. 10

    Pith/arXiv arXiv 2019

  30. [30]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal Policy Optimization Algorithms, Aug. 2017. URLhttp://arxiv.org/abs/1707.06347. arXiv:1707.06347 [cs]. 11 A Appendix A.1 Simulation Parameters Table 1 displays the default parameters used when no domain randomization is used. The domain randomization used in this work is centered on V...

    Pith/arXiv arXiv 2017