REVIEW 3 major objections 7 minor 22 references
SMaRCSim: Maritime Robotics Simulation Modules
T0 review · 3 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper presents SMaRCSim, an open-source Unity-based maritime simulation suite that claims to combine multi-domain mixed-fidelity simulation, integrated command-and-control mission planning, and faster-than-real-time reinforcement…
desk verdict A useful open-source maritime simulator contribution whose headline RL speedup needs a measurement protocol before the performance claims can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying mechanism is the Unity game engine used as the base: its prefab system lets researchers assemble new vehicles and environments from existing objects without deep source changes, its PhysX physics backend accepts hand-computed forces on rigid bodies, and its machine-learning interface exposes the simulation loop directly to Python for reinforcement learning. On top of this, a ROS 2/MQTT bridge and an extensible multi-domain command-and-control API connect simulation to real field operations, and Unity's native support for simulation speeds up to 100x real time provides the headroom behind the ~50x reinforcement-learning speed figure.
What would settle it
Measure the wall-clock-to-simulated-time ratio for the paper's own 64-AUV reinforcement-learning scene on the same hardware and check whether the ROS 2 connection between the command-and-control interface and a simulated vehicle stays alive; a ratio far below 50x or a broken C2 link would refute the core integration claim.
Extended reading notes
Core claim
SMaRCSim's central claim is that one simulation stack can cover the full lifecycle of maritime robotics research: multi-domain scenes with aerial, surface, and underwater vehicles simulated together; mixed-fidelity setups where, say, sensors are rendered more accurately than vehicle dynamics; an integrated command-and-control mission planner that works with 3D waypoints and imported bathymetry; and a faster-than-real-time mode, reported at about 50x, for reinforcement learning through Unity's low-level Python API. The authors would state this as a solved engineering integration problem rather than a new algorithm: the same code and interfaces used in simulation connect to real vehicles through ROS 2 and MQTT, with custom physics forces for control surfaces, currents, and winds added directly to Unity's PhysX rigid bodies.
Load-bearing premise
The argument's load-bearing premise is that the reported ~50x faster-than-real-time speed and the integrated command-and-control workflows hold up in ordinary academic use, since the paper states these capabilities without giving a hardware specification, a workload description, or a measurement protocol.
Editorial extensions
If this is right
- A project can develop and test controllers, train RL policies, and plan field missions inside one toolchain, eliminating simulator-switching costs.
- RL-trained underwater, surface, and aerial policies can be deployed through the same ROS 2 interfaces used in simulation, easing the transfer path.
- Multi-domain operations such as an aerial vehicle recovering an AUV or coordinating with a surface vessel become testable in a single scene.
- Field-collected bathymetry can be dragged into the command-and-control scene immediately, making mission planning possible at remote sites without a server connection.
- Custom hydrodynamics and environmental disturbances can be added without rewriting the engine, supporting a sim-to-real-to-sim residual correction loop.
Reading between the lines
- If the reported ~50x real-time speed survives a reproducible benchmark, SMaRCSim would close a gap between high-fidelity simulators and lightweight RL simulators; until then it is best read as an integration claim rather than a measured result.
- The sim-to-real-to-sim loop suggests a practical workflow where learned residual dynamics from real trials are inserted back into the Unity simulation, which could make simulation-backed field planning more reliable.
- A natural test is to run a standard RL control task such as station-keeping or path following in SMaRCSim and in one existing simulator on identical hardware to verify speed and learning efficiency.
- Because the suite is open-source, the field could adopt it as a shared benchmark for multi-domain maritime scenarios, which existing single-domain simulators do not support.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper presents SMaRCSim, a set of open-source Unity-based simulation modules developed at SMaRC for maritime robotics. The authors motivate the tool by three use cases: multi-domain mixed-fidelity simulation, integration with command-and-control (C2) software for field operations, and faster-than-real-time simulation for reinforcement learning. The main claims are that SMaRCSim supports simultaneous simulation of aerial, surface, and underwater vehicles; provides an integrated C2 interface through WARA-PS; and enables RL at roughly 50 times real-time speed. The paper includes a comparison table of existing simulators, a system description with three application scenarios, and a link to the public GitHub repository.
Significance. If the claims hold, SMaRCSim would fill a genuine gap: an open-source, extensible, multi-domain maritime simulator with ROS 2 support, integrated mission planning, and faster-than-real-time RL capability. The public GitHub repository and the authors' concrete field-experience background are strengths; the tool is likely to be reused by the community if the code is well documented. However, the paper's value proposition rests heavily on the quantitative 'RL feasible (Yes, ~50xRT)' claim in Table I and Section III-C, which is not supported by any measurement or benchmark data in the manuscript. The paper also lacks an evaluation section entirely, providing no verification that the described modules function as advertised. These gaps are load-bearing because the stated motivation is computational efficiency for learning-based methods. The paper is a useful software description, but it needs a reproducible performance evaluation before the central claims can be accepted.
major comments (3)
- [Section III-C, Table I] The headline capability 'RL feasible (Yes, ~50xRT)' is asserted without any measurement. No hardware configuration, scene complexity, number of vehicles, episode length, or Unity time-scale setting is provided, and the footnote in Section III-C ('Subject to CPU resources and complexity of the particular simulation') only reinforces that the figure is context-dependent. Because the abstract and Fig. 1c advertise faster-than-real-time RL, and the Table I comparison is the main quantitative basis for the tool's value proposition, the authors must report a reproducible benchmark: for example, a fixed scene with N agents, the exact Unity and physics settings, wall-clock versus simulation time over a defined episode, and the median and spread over multiple runs. This benchmark should also be used to substantiate the '~50xRT' entry or to correct it if that is only an optimistic upper bound.
- [Whole manuscript] The paper contains no experiments or evaluation section. The claims of multi-domain mixed-fidelity support, C2 integration, and Real-2-Sim are supported only by static images in Fig. 1 and by anecdotal descriptions. A software paper should at least provide a minimal reproducible example, such as a launch script and a one-vehicle or multi-vehicle scenario, together with quantitative evidence that the modules run as described (e.g., vehicle trajectories, sensor messages, C2 commands). Consider adding a small demonstration scenario with measurable outputs to validate the architecture and to show that the tool is usable by others.
- [Table I] The comparison methodology for Table I is not transparent. The 'RL feasible' entries for HoloOcean ('Partial (~2xRT)') and MARUS ('Partial') are not cited or measured, and the SMaRCSim entry 'Yes (~50xRT)' appears to be inferred from Unity's native 100x real-time capability mentioned in Section III-C, not from a SMaRCSim-specific measurement. State explicitly which entries are measured by the authors, which are taken from the respective public repositories, and which are from the literature. Without this distinction, the table cannot be independently checked and the comparison is not meaningful.
minor comments (7)
- [Software Release section] The GitHub link points to 'smarc-project/smarc2', but the paper's name is SMaRCSim; clarify the relationship between the repository name and the paper title, and consider adding a release tag or a DOI for versioning and reproducibility.
- [Table I] There is a typo: 'Inaccesible' should be 'Inaccessible'. Also, the phrase 'RL feasible (Yes, ~50xRT)' uses 'xRT' without defining the term; state that RT denotes the real-time factor (simulation time divided by wall-clock time) near the table or in the text.
- [Section III-C] The acronym 'LLAPI' is used without expansion; the authors should write 'Low-Level API (LLAPI)' on first use, and clarify the relationship between the MLAgents platform and the Unity engine.
- [Fig. 1c] The caption claims '64 AUVs running faster than real-time' but provides no quantitative speedup; give the measured factor in the caption or refer to the benchmark requested in Major Comment 1.
- [Section IV] The conclusion states that SMaRCSim 'enables the investigation of learning-based methods for control through its computational efficiency'; this is stronger than the evidence supports, as no RL experiment is reported anywhere in the paper.
- [References and text] The names 'DA VE' and 'W A VE' appear with inserted spaces in the body and in the reference titles; these should be written as 'DAVE' and 'WAVE'. The author name 'Karta ˇsev' in the byline also contains a corrupted character.
- [Table I] For the 'Latest update' row, the SMaRCSim entry is given as 2025, but the paper does not state the commit date of the repository used; specify the date of the last commit or a release version so the entry is checkable.
Circularity Check
No significant circularity: SMaRCSim is a software-description paper with no derivation-to-prediction chain; the unverified ~50xRT figure is an evidentiary gap, not a circular argument.
full rationale
The paper describes an open-source Unity-based maritime simulation package and compares its features with other simulators. It contains no mathematical derivation, no fitted parameters, and no prediction that reduces to its own inputs. The claim 'RL feasible (Yes, ~50xRT)' in Table I and Section III-C is a performance assertion, not a derived result; the paper itself hedges it with footnote 3: 'Subject to CPU resources and complexity of the particular simulation.' An unsupported quantitative claim is a correctness or reproducibility weakness, but it is not circular because the figure is not defined in terms of the claim it is supposed to support, nor fitted to the outcome being predicted. The self-citations [2] and [3] are prior field-experiment papers used to motivate the need for flexible C2 software ('In our experience [2, 3], existing C2 software lacks the flexibility...'); they are not invoked as evidence that SMaRCSim works, and the central contribution is the released simulator itself, which is externally checkable at the provided GitHub repository. No load-bearing argument reduces to a self-citation, and no equation is derived from another. The paper is therefore self-contained as a software overview, and the appropriate circularity score is 0.
Assumptions & free parameters
assumptions (2)
- domain assumption Unity's physics engine can be augmented with manually computed forces (e.g., control surfaces, currents, wind) to reach acceptable maritime simulation fidelity.
- domain assumption The performance figures in Table I, such as ~50xRT, are representative of typical use.
Cite this review
Pith. "Pith review of SMaRCSim: Maritime Robotics Simulation Modules." pith.science (2026). https://pith.science/paper/CBEAGTJR
@misc{pith2026250607781,
author = {Pith},
title = {Pith review of: SMaRCSim: Maritime Robotics Simulation Modules},
year = {2026},
howpublished = {\url{https://pith.science/paper/CBEAGTJR}},
note = {Machine review of arXiv:2506.07781}
}
read the original abstract
Developing new functionality for underwater robots and testing them in the real world is time-consuming and resource-intensive. Simulation environments allow for rapid testing before field deployment. However, existing tools lack certain functionality for use cases in our project: i) developing learning-based methods for underwater vehicles; ii) creating teams of autonomous underwater, surface, and aerial vehicles; iii) integrating the simulation with mission planning for field experiments. A holistic solution to these problems presents great potential for bringing novel functionality into the underwater domain. In this paper we present SMaRCSim, a set of simulation packages that we have developed to help us address these issues.
Figures
Reference graph
Works this paper leans on
-
[1]
Abdelhakim Amer, Olaya ´Alvarez Tu˜n´on, Halil ˙Ibrahim U˘gurlu, Jonas Le Fevre Sejersen, Yury Brodskiy, and Erdal Kayacan. UNav-Sim: A Visually Realistic Under- water Robotics Simulator and Synthetic Data-Generation Framework. In2023 21st International Conference on Advanced Robotics (ICAR), pages 570–576. doi: 10.1109/ICAR58858.2023.10406819
-
[2]
To- wards a Cyber-Physical System for Hydrobatic AUVs
Sriharsha Bhat, Ivan Stenius, Nils Bore, Josefine Sev- erholt, Carl Ljung, and Ignacio Torroba Balmori. To- wards a Cyber-Physical System for Hydrobatic AUVs. InOCEANS 2019 - Marseille, pages 1–7, . doi: 10.1109/OCEANSE.2019.8867392
-
[3]
A Cyber-Physical System for Hy- drobatic AUVs: System Integration and Field Demon- stration
Sriharsha Bhat, Ignacio Torroba, ¨Ozer ¨Ozkahraman, Nils Bore, Christopher Iliffe Sprague, Yiping Xie, Ivan Ste- nius, Josefine Severholt, Carl Ljung, John Folkesson, and Petter ¨Ogren. A Cyber-Physical System for Hy- drobatic AUVs: System Integration and Field Demon- stration. In2020 IEEE/OES Autonomous Underwa- ter V ehicles Symposium (AUV), pages 1–8, ...
-
[4]
Development of an Under- water Simulator using Unity3D and Robot Operat- ing System
Akash Chaudhary, Rajat Mishra, Bharath Kalyan, and Mandar Chitre. Development of an Under- water Simulator using Unity3D and Robot Operat- ing System. InOCEANS 2021: San Diego – Porto, pages 1–7. ISBN 978-0-692-93559-0. doi: 10.23919/OCEANS44145.2021.9706012
-
[5]
Patryk Cie ´slak. Stonefish: An Advanced Open-Source Simulation Tool Designed for Marine Robotics, With a ROS Interface — IEEE Conference Publication — IEEE Xplore. doi: 10.1109/OCEANSE.2019.8867434
- [6]
-
[7]
Simu2VITA: A General Purpose Underwater Vehicle Simulator
prefix=de useprefix=true family=Cerqueira Gava, given=Pedro Daniel, Cairo L ´ucio Nascimento J ´unior, Juan Ram ´on Belchior de Franc ¸a Silva, and Geraldo Jos´e Adabo. Simu2VITA: A General Purpose Underwater Vehicle Simulator. 22(9):3255. ISSN 1424-8220. doi: 10.3390/s22093255
-
[8]
Fossen.Handbook of Marine Craft Hydrodynam- ics and Motion Control, 2nd Edition
Thor I. Fossen.Handbook of Marine Craft Hydrodynam- ics and Motion Control, 2nd Edition. . ISBN 978-1-119- 57503-0
Show all 22 references
-
[9]
Thor I. Fossen. Cybergalactic/PythonVehicleSimulator,
-
[10]
Thor I. Fossen. Cybergalactic/MSS,
-
[11]
Unity: A general platform for intelligent agents
Arthur Juliani, Vincent-Pierre Berges, Ervin Teng, An- drew Cohen, Jonathan Harper, Chris Elion, Chris Goy, Yuan Gao, Hunter Henry, Marwan Mattar, and Danny Lange. Unity: A general platform for intelligent agents. arXiv preprint arXiv:1809.02627, 2020
2020 arXiv
-
[12]
Open Source Simulator for Un- manned Underwater Vehicles using ROS and Unity3D
Pushkal Katara, Mukul Khanna, Harshit Nagar, and An- napurani Panaiyappan. Open Source Simulator for Un- manned Underwater Vehicles using ROS and Unity3D. In2019 IEEE Underwater Technology (UT), pages 1–7. doi: 10.1109/UT.2019.8734309
-
[13]
Champion-level drone racing using deep re- inforcement learning.Nature, pages 982–987, August
Elia Kaufmann, Leonard Bauersfeld, Antonio Loquercio, Matthias M ¨uller, Vladlen Koltun, and Davide Scara- muzza. Champion-level drone racing using deep re- inforcement learning.Nature, pages 982–987, August
-
[14]
MARUS - A Ma- rine Robotics Simulator
Ivan Loncar, Juraj Obradovic, Natko Krasevac, Luka Mandic, Igor Kvasic, Fausto Ferreira, Vladimir Slosic, Dula Nad, and Nikola Miskovic. MARUS - A Ma- rine Robotics Simulator. InOCEANS 2022, Hampton Roads, pages 1–7. ISBN 978-1-66546-809-1. doi: 10.1109/OCEANS47191.2022.9976969
2022
-
[15]
Robot operat- ing system 2: Design, architecture, and uses in the wild.Science Robotics, 7(66):eabm6074, 2022
Steven Macenski, Tully Foote, Brian Gerkey, Chris Lalancette, and William Woodall. Robot operat- ing system 2: Design, architecture, and uses in the wild.Science Robotics, 7(66):eabm6074, 2022. doi: 10.1126/scirobotics.abm6074
2022 doi
-
[16]
Scherer, Martin V oss, Luiz Ricardo Douat, and Thomas Rauschen- bach
Musa Morena Marcusso Manh ˜aes, Sebastian A. Scherer, Martin V oss, Luiz Ricardo Douat, and Thomas Rauschen- bach. UUV Simulator: A Gazebo-based package for underwater intervention and multi-robot simulation. In OCEANS 2016 MTS/IEEE Monterey, pages 1–8. doi: 10.1109/OCEANS.201...
2016
-
[17]
Pereira, J
F.L. Pereira, J. Pinto, J.B. Sousa, R.M.F. Gomes, G.M. Goncalves, P.S. Dias, F.L. Pereira, J. Pinto, J.B. Sousa, R.M.F. Gomes, G.M. Goncalves, and P.S. Dias. Mission planning and specification in the Neptus framework. InProceedings ICRA 2006., pages 3220–3225. doi: 10.1109/ROB...
2006 arXiv
-
[18]
Mangelson
Easton Potokar, Spencer Ashford, Michael Kaess, and Joshua G. Mangelson. HoloOcean: An Un- derwater Robotics Simulator. In2022 Interna- tional Conference on Robotics and Automation (ICRA), pages 3040–3046. ISBN 978-1-72819-681-7. doi: 10.1109/ICRA46639.2022.9812353
-
[19]
Javier Fernandez, and Pe- dro J
Mario Prats, Javier Perez, J. Javier Fernandez, and Pe- dro J. Sanz. An open source tool for simulation and supervision of underwater intervention missions. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems, pages 2577–2582. ISBN 978- 1-4673-1736-8 97...
2012
-
[20]
Hollinger, Aaron Marburg, Madison Pickett, and Joseph R
Marcus Rosette, Hannah Kolano, Chris Holm, Geof- frey A. Hollinger, Aaron Marburg, Madison Pickett, and Joseph R. Davidson. W A VE: An open-source underWater Arm-Vehicle Emulator. In2024 IEEE International Conference on Robotics and Automation (ICRA), pages 1505–1511. doi: 10....
-
[21]
Zhang, Woen-Sug Choi, Jessica Herman, Du- ane Davis, Carson V ogt, Michael McCarrin, Yadunund Vijay, Dharini Dutia, William Lew, Steven Peters, and Brian Bingham
Mabel M. Zhang, Woen-Sug Choi, Jessica Herman, Du- ane Davis, Carson V ogt, Michael McCarrin, Yadunund Vijay, Dharini Dutia, William Lew, Steven Peters, and Brian Bingham. DA VE Aquatic Virtual Environment: Toward a General Underwater Robotics Simulator
- [2023]
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.