{"id":"b838d47c-885f-4b33-bab4-e03cdb11be89","arxiv_id":"2512.08656","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"A DRL velocity controller for AUVs, trained in under 3 minutes in simulation, transfers zero-shot to a real underwater vehicle for agile 6DOF path-following.","lead":"This paper trains an underwater-robot controller entirely in simulation for about three minutes, then runs the same trained policy on a real BlueROV2 without re-tuning to follow paths and hold unusual orientations. The potential benefit is avoiding time-consuming manual controller tuning whenever an AUV's payload or configuration changes.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Zero-shot claim is under-supported: pool validation has no quantitative error metrics, repeated trials, or baseline comparisons, and the paper itself reports unquantified tracking deviations in sway, heave, and attitude.","rationale":"The reader's weakest assumption — that the simulation model with its limited domain randomization is a sufficient proxy for real-world hydrodynamics, thrust allocation, and disturbances — is indeed the deepest risk to the zero-shot claim. I sharpen this into an evidence-based concern: the paper provides no quantitative validation that this assumption actually holds in the pool trials. The three trials are single runs with qualitative descriptions and self-admitted deviations (Figure 4b: sway/heave lag; Figure 4c: heave lag causing depth deviation). Thus the central claim that the policy achieves 'accurate' tracking and 'robust zero-shot control' is not established by the reported data. The reward-curve inconsistency (Eq. 5 vs Figure 3) further weakens the training-time claim. These concerns do not prove the method false; they show the paper's support is insufficient. The reader's CONDITIONAL verdict already reflects this, so I do not move the verdict. If the authors supply the requested quantitative metrics and clarify the reward discrepancy, the concerns could be resolved.","tokens_in":8662,"tokens_out":7483,"duration_ms":82095,"concrete_test":"From the raw logs of the three pool trials, compute per-DOF RMS tracking error and steady-state offset for each reference segment (constant and time-varying), then repeat each configuration at least 5 times and report mean and 95% confidence intervals. Compare these errors against (a) a pre-specified acceptable threshold (e.g., 5% of the reference speed for velocities and 5° for attitude) and (b) the same policy's errors in the simulation environment under identical reference trajectories. If real-world errors exceed the threshold or are significantly larger than simulation errors, the zero-shot sim-to-real claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that a simulation-trained policy transfers zero-shot to an agile 6DOF velocity controller on a real AUV. The only evidence for this is Section 5.2's three single pool trials. These trials are presented as time histories with qualitative descriptions ('closely tracks', 'appears robust', 'unable to track fast variations', 'slower response in heave') rather than as measured tracking errors. No RMS/MAE errors, no repeated runs, no confidence intervals, and no comparison to a baseline controller (e.g., the PID the authors argue against) are provided. The paper itself admits nontrivial deviations: in the ballast trial, sway and heave fail to track fast variations; in the random-orientation trial, heave lag causes the depth to deviate from the desired path; and attitude offsets are attributed to imperfect thrust allocation. Without quantitative metrics, the reader cannot distinguish 'robust zero-shot control' from acceptable-but-marginal performance or from trials selected to look favorable. This is load-bearing because the zero-shot claim is the paper's main contribution: if the real tracking errors are large or unrepeatable, the claim collapses. Additionally, Figure 3's reported final mean reward of 315 is inconsistent with Eq. 5, whose per-step maximum is approximately 0.95 (sum of weights = 0.4 + 0.05 + 0.2 + 0.3), or at most ~47.5 over a 5 s episode at 10 Hz; this undermines the reported '3-minute convergence' evidence unless the reward definition is clarified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents Sim2Swim, a reinforcement-learning-based controller for holonomic AUVs that outputs desired forces/torques for velocity and attitude tracking. Trained in Isaac Lab with PPO and domain randomization, the policy uses integral observations of velocity and orientation errors to reduce steady-state error. The authors claim zero-shot sim2real transfer after only 3 minutes of training, and validate with three pool trials on a BlueROV2 Heavy: straight-line path following, a ballast perturbation, and a square path with random attitude setpoints. The central claim is that the trained policy generalizes across payloads and can perform agile 6DOF maneuvering without any real-world fine-tuning.","tokens_in":9125,"tokens_out":4024,"duration_ms":40718,"significance":"If supported, this would be a meaningful step toward practical DRL for AUV control: fast training, no per-platform retuning, and demonstrated agility in attitude and velocity tracking. The proposed integral-observation mechanism is a plausible approach to remove steady-state error observed in prior work, and the choice of a force/torque action space rather than thruster-level commands is a sensible design decision that maintains platform generality. Hardware validation in three configurations is a useful contribution, provided the performance is quantified. The paper also brings together recent work on massively parallelized RL and AUV sim2real.","major_comments":[{"comment":"The reported mean reward of 315 in the final learning iteration is inconsistent with the reward definition. Per step, each term in Eq. (5) is no greater than its weight; with Table 1 weights (0.4+0.05+0.2+0.3), the maximum per-step reward is 0.95. With a 5 s episode at 10 Hz, the maximum episode reward is 47.5, not 315. Unless there is an omitted scaling, discounting, or a different reward normalization, Figure 3 contradicts Eq. (5). This undermines the '3-minute convergence' evidence and must be resolved before the training claim is credible.","section":"§5.1, Figure 3, Eq. (5)–(8)"},{"comment":"The zero-shot claim rests on qualitative descriptions of three single pool trials. No tracking-error metrics (e.g., RMS or MAE of velocity and attitude errors), no repeated trials, and no confidence intervals are reported. The text itself admits nontrivial deviations: 'unable to track fast variations' in sway and heave, 'slower response in heave', a depth deviation from the desired path, and attitude offsets attributed to thrust allocation. Without quantitative performance measures, the reader cannot distinguish robust zero-shot control from marginal or trial-selected behavior. This is load-bearing for the paper's central contribution and should be addressed with quantitative error metrics and at least repeated runs.","section":"§5.2, Figure 4"},{"comment":"Domain randomization ranges for mass, volume, and CB–CM offset are not specified. The paper states only that mass and volume are 'uniformly sampled' and the CB–CM offset is 'uniformly sampled in a sphere'. To assess whether the real ballast perturbation (600 g, ~5% mass change, buoyancy sign change) is covered by the randomization, the ranges are essential. Without them, the transferability argument is not reproducible and the zero-shot claim is under-supported. Please provide the exact distributions and justify their coverage of the real parametric variation.","section":"§4.4, §5.2 (ballast trial)"},{"comment":"The paper advocates against hand-tuned PID controllers but provides no baseline comparison to a tuned PID or to prior DRL controllers (e.g., Cai et al. 2025 or Sufán and Troni 2025). The claim that Sim2Swim shows 'superior performance' or 'robust control' is not substantiated without a comparative baseline under the same test conditions. Adding a baseline, even for a subset of trials, would place the results in context and support the 'general' claim.","section":"§5.2, §6"}],"minor_comments":[{"comment":"The sentence describing the policy architecture is incomplete: 'The policy is realized as a 2-layer MLP with, and is trained...'. Please specify the hidden dimension and activation function.","section":"§4.2"},{"comment":"The row 'Actions w_i 0.3' presumably refers to the action reward weight w_a, not the integral-state weight. Please correct the notation.","section":"Table 1"},{"comment":"The notation r_i and w_i is ambiguous: Eq. (6) defines r_i generically, but the terms for orientation, angular velocity, and linear velocity all use different weights. It would be clearer to write r_omega, r_v, and separate r_q or explain how the three r_i terms are combined.","section":"Eq. (5)–(8)"},{"comment":"The attitude subplot labels appear garbled (e.g., '3A', '?d3dAd'). The roll/pitch/yaw curves should be clearly identified. Also, the time axes are not aligned across rows, making comparison difficult.","section":"Figure 4"},{"comment":"The desired speed is fixed at V_d = 0.5 m/s during training. The pool trials appear to use similar speeds, but a statement on how well the policy performs outside this training speed would strengthen the generality claim.","section":"§4.5"},{"comment":"Please state whether the three trials are single runs or representative of multiple runs, and report the number of trial repetitions, if any.","section":"§5.2"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a relevant and timely problem, but the evidence at present is not sufficient for acceptance. The reward inconsistency in Figure 3 is a concrete red flag that must be fixed, and the qualitative experimental validation without metrics or baselines falls short of the journal's standard for a zero-shot sim2real claim. The manuscript needs a revision with quantitative results, repeated trials, and full specification of the domain randomization. The 'first' claim should also be carefully positioned against Sufán and Troni (2025), who performed sim2real 6-DOF setpoint regulation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this paper extends massively parallel DRL from position setpoint to time-varying 6-DOF velocity/attitude tracking, and the 3-minute training time is impressive. The pool trials are real hardware demos with a ballast offset and extreme poses, which is more than many sim-only papers do. But the central zero-shot claim rests on three single runs, reported with qualitative descriptions and no error metrics, baselines, or repeated trials. That is the main soft spot.\n\nWhat's new: Cai et al. (2025) did position setpoint regulation; Sim2Swim adds integral observations, targets time-varying velocity and orientation references, and converges in under 3 minutes on a laptop GPU. The integral action plausibly addresses the steady-state sway/pitch errors Cai reported. The pool validation includes a 600 g ballast that flips the vehicle's buoyancy, and a random-orientation trial with ±90° pitch/roll. That's a meaningful step forward.\n\nThe paper is well organized, and the authors are honest about limitations: they explicitly say sway and heave can't track fast variations and that thrust-allocation imperfections cause attitude offsets. They don't oversell perfect tracking.\n\nThe problems are all in the evidence quality. No RMS or MAE errors, no confidence intervals, no comparison against a tuned PID or the prior DRL controller. The heave lag in Figure 4c causes a visible depth deviation, but we have no idea how large it is in meters. For a \"zero-shot sim2real agility\" claim, that's load-bearing. Also missing: repeated trials, code/data release, and a statement of the simulation control frequency. The reported mean reward of 315 looks suspicious if you assume 10 Hz, but it's plausible at 50 Hz or higher; the paper just never says. The domain randomization is narrow—mass, volume, and CB–CM offset only—no hydrodynamic coefficient or sensor noise variation.\n\nThe stress-test note's reward inconsistency is probably a false alarm, but it's a fair catch that the paper omits the time step. The bigger critique stands.\n\nWho should read it: people working on sim2real RL for marine robots and anyone interested in ultra-fast policy training. It deserves peer review—the method is sound and the training speed is a useful result—but it needs major revision: add quantitative tracking metrics, baselines, repeated experiments, and release code and training schedules.","headline":"Sim2Swim is a plausible and genuinely fast DRL velocity controller for AUVs, but its zero-shot claim is under-supported by purely qualitative pool trials; worth a serious referee but needs quantitative validation before I'd trust it.","tokens_in":9561,"tokens_out":3873,"would_cite":true,"duration_ms":38217,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a policy trained entirely in simulation in under three minutes can control a real AUV's six-degree-of-freedom velocity and attitude with zero real-world tuning.","keywords":["Underwater robotics","Deep reinforcement learning","Zero-shot sim2real","Velocity control","6DOF maneuvering","Domain randomization","AUV control","Integral action"],"falsifier":"Deploy the trained policy on the same ROV while a controlled current or wave disturbance is applied and measure whether velocity and quaternion errors return to zero once references stabilize; if they remain biased or oscillate under a disturbance within the vehicle's rated operating envelope, the zero-shot claim fails. A second, simpler test is to apply the policy to a different ROV with a different thruster layout without retraining—if tracking degrades, the controller is not platform-general.","tokens_in":8602,"feed_emoji":"🤖","tokens_out":4971,"duration_ms":54433,"temperature":0.7,"pith_summary":"The paper is trying to establish that a deep reinforcement learning controller for a holonomic AUV can be trained entirely in simulation, in about three minutes on a laptop, and then deployed on the real vehicle with zero additional tuning. The controller tracks six-degree-of-freedom references: desired linear velocities in surge, sway, and heave, plus arbitrary orientation references. This is more demanding than station-keeping because both translation and rotation vary over time. The authors argue that if this works, marine operators can stop hand-tuning PID gains for every payload change and instead use one simulation-trained policy across configurations. They support the claim with pool experiments showing accurate velocity and attitude tracking, including with a 600-gram asymmetric ballast and extreme roll and pitch commands.","feed_headline":"3-minute sim training yields zero-shot AUV control","feed_subtitle":"One policy, trained in simulation, tracks agile 6DOF paths in pool trials with no re-tuning.","key_machinery":"A two-layer MLP policy trained with proximal policy optimization in thousands of parallel, short (5-second) simulated episodes. Its observation vector combines quaternion error, linear velocity error, angular velocity, and integral states of velocity and attitude error; the integral states give the policy memory and remove steady-state error. Actions are six normalized body-frame force/torque commands mapped through the thrust-gain matrix K, so the policy need not learn thruster allocation. Domain randomization perturbs mass, volume, and the center-of-buoyancy offset relative to the center of mass to force transferable behavior.","core_discovery":"Sim2Swim's central claim is that integral action appended to a learned velocity/attitude policy eliminates the steady-state errors seen in earlier DRL position controllers while preserving fast, robust convergence. The policy observes quaternion error, linear velocity error, angular velocity, and accumulated integrals of velocity and attitude error; outputs six normalized force/torque commands which are scaled by the vehicle's thrust-gain matrix; and is trained with domain randomization over mass, volume, and the offset between center of buoyancy and center of mass. In pool trials on an instrumented ROV, the same zero-shot policy tracks straight-line paths, a square path under an added 600 g","pith_inferences":["If the recipe generalizes, any holonomic vehicle with six actuated DOFs — not just this ROV — could inherit the same zero-shot pipeline, needing only a randomized rigid-body model.","The decisive next test is environmental disturbance: currents and waves are absent from the 5-second training episodes, so the claim of generality beyond calm pools is unproven.","The authors' 3-minute budget suggests sample efficiency is no longer the bottleneck; simulator fidelity and the coverage of domain randomization are what determine where zero-shot transfer ends."],"forward_implications":["Controller retuning between deployments becomes unnecessary: a single policy trained once in simulation carries over across payloads within the randomized range.","Attitude can be commanded independently of path, enabling inspection maneuvers where the vehicle holds a fixed orientation to a structure while translating along it.","Integral observations close the steady-state velocity and pitch/sway errors that earlier learning-based AUV controllers left behind.","Reducing training from hours to minutes on a laptop makes DRL-based control a realistic option for small field robots without specialized infrastructure."],"fun_headline_variants":["3-minute sim yields zero-shot AUV 6DOF agility","Zero-shot AUV control from 3-minute DRL training","AUVs skip re-tuning with 3-minute sim policy","3 sim minutes train AUVs for agile zero-shot motion"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that a rigid-body simulator with randomized mass, volume, and buoyancy offset, and 5-second episodes, captures enough of the real vehicle's hydrodynamics and disturbances that a policy trained only in simulation transfers to the pool without any real-world adjustment.","fun_headline_variants_meta":{"raw":{"variants":["3-minute sim yields zero-shot AUV 6DOF agility","Zero-shot AUV control from 3-minute DRL training","AUVs skip re-tuning with 3-minute sim policy","3 sim minutes train AUVs for agile zero-shot motion"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000237,"raw_usage":{"total_tokens":1345,"prompt_tokens":745,"completion_tokens":600,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":489,"completion_tokens_details":{"reasoning_tokens":528}},"tokens_in":489,"tokens_out":600,"duration_ms":6446,"temperature":1.0,"reasoning_tokens":528,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T17:35:02.784836+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Deploy the trained policy on the same ROV while a controlled current or wave disturbance is applied and measure whether velocity and quaternion errors return to zero once references stabilize; if they remain biased or oscillate under a disturbance within the vehicle's rated operating envelope, the zero-shot claim fails. A second, simpler test is to apply the policy to a different ROV with a different thruster layout without retraining—if tracking degrades, the controller is not platform-general.","supporting_citations":[],"review_version":1}