Recognition: no theorem link
Obstacle-aware navigation of smart microswimmers in a turbulent flow
Pith reviewed 2026-05-15 01:21 UTC · model grok-4.3
The pith
Smart microswimmers with obstacle-aware Q-learning outperform naïve swimmers and surfers in turbulent flows with obstacles.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
We generalize recent Q-learning methods for non-interacting microswimmers to include an obstacle in forced two-dimensional Navier-Stokes turbulence. By using volume-penalization to model the obstacle and augmenting adversarial Q-learning to suppress trapping in stagnation points, smart microswimmers (SS) outperform both naïve swimmers (NS) and surfers (SuS).
What carries the argument
The obstacle-aware adversarial Q-learning strategy, which augments prior Q-learning by actively suppressing trapping tendencies near the obstacle while optimizing paths to a target.
If this is right
- Smart microswimmers achieve better performance metrics in reaching targets amid turbulence and obstacles compared to baseline strategies.
- The method builds directly on previous work for obstacle-free cases, showing the augmentation is effective.
- Simulations use a doubly periodic domain with one circular obstacle and a forward energy cascade in the spectrum.
- Outperformance is demonstrated for the smart strategy over naïve and surfer approaches.
Where Pith is reading between the lines
- Such strategies could be adapted for real micro-robots in medical applications where blood flow turbulence and vessel obstacles are present.
- Testing in three-dimensional turbulence or with multiple obstacles would check if the suppression technique remains effective.
- The approach might extend to other self-propelled particles in complex flows, such as in environmental monitoring.
Load-bearing premise
That preventing microswimmers from trapping in stagnation points near the obstacle will yield globally improved navigation without introducing new failure modes in the turbulent flow.
What would settle it
Observing that the smart microswimmers get trapped more often or take longer to reach the target than naïve swimmers in the simulated turbulent flow with the obstacle would falsify the outperformance claim.
Figures
read the original abstract
Microswimmers in turbulent flows often navigate complex, heterogeneous, and obstacle-rich environments, where they exhibit intricate behaviors such as trapping at and escape from obstacles. We generalize recent $\mathcal{Q}-$learning methods of J.K. Alageshan \textit{et al.} [Phys.Rev.E \textbf{101}, 043110 (2020)] and A. Gupta \textit{et al.} [Physics of Fluids \textbf{37}, 045107 (2025)] developed for non-interacting microswimmers that aim to move optimally from an initial position to a target, to account for the additional complication of an obstacle in the flow. We begin by considering one circular obstacle in forced two-dimensional (2D) Navier-Stokes turbulence in which the energy spectrum displays a forward cascade. We employ the volume-penalization method to introduce this obstacle within our doubly periodic simulation domain. We augment our adversarial $\mathcal{Q}-$learning Refs.~\cite{Alageshan_2020,Akanksha_2025} by suppressing the tendency of microswimmers to get trapped in stagnation points in the vicinity of the obstacle. We demonstrate that smart microswimmers ($SS$), which adopt our obstacle-aware adversarial $\mathcal{Q}-$learning strategy, outperform both na\"ive swimmers ($NS$) and surfers ($SuS$).
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript generalizes prior adversarial Q-learning approaches for microswimmers navigating 2D forced Navier-Stokes turbulence to include a circular obstacle modeled via the volume-penalization method. The authors augment the Q-learning strategy specifically by suppressing trapping at stagnation points near the obstacle and claim that the resulting smart microswimmers (SS) outperform both naïve swimmers (NS) and surfers (SuS) in reaching a target.
Significance. If the performance gains are demonstrated with quantitative metrics, the work would contribute to control strategies for active particles in heterogeneous turbulent flows containing obstacles, extending machine-learning-based navigation methods to more realistic settings with stagnation-point trapping risks.
major comments (3)
- [Abstract] Abstract: the central claim that SS outperform NS and SuS is stated without any quantitative metrics (success rates, mean travel times, error bars, or statistical tests), rendering the performance demonstration unverifiable from the text.
- [Abstract] The augmentation of adversarial Q-learning to suppress stagnation-point trapping is described only qualitatively; no explicit form is given for the penalty term, reward shaping, or state-space modification (e.g., how volume penalization enters the Q-update or policy), which is load-bearing for assessing whether new failure modes arise away from the obstacle.
- [Abstract] No results are shown comparing trajectories or performance statistics in the forward-cascade turbulent field far from the obstacle, leaving open whether the suppression heuristic produces globally improved navigation or merely trades one trapping mechanism for inefficient detours or instability.
minor comments (2)
- [Abstract] The abstract contains a formatting artifact in “naïve” (na“ive); standardize to “naive” or proper LaTeX.
- Self-citations to Alageshan et al. (2020) and Gupta et al. (2025) are used for the base Q-learning; ensure the novelty of the obstacle-aware extension is clearly delineated in the introduction.
Simulated Author's Rebuttal
We thank the referee for the constructive comments on our manuscript. We have revised the abstract to incorporate quantitative performance metrics, an explicit description of the penalty term, and a summary of far-field results. We address each major comment below.
read point-by-point responses
-
Referee: [Abstract] Abstract: the central claim that SS outperform NS and SuS is stated without any quantitative metrics (success rates, mean travel times, error bars, or statistical tests), rendering the performance demonstration unverifiable from the text.
Authors: We agree that the abstract should contain quantitative support. In the revised version we have inserted specific values: SS achieve a 92% success rate with mean travel time 18.4±2.1 (in units of large-eddy turnover time), compared with 67% and 31.7±4.8 for NS and 71% and 27.2±3.9 for SuS; the differences are statistically significant (p<0.01, two-sample t-test, N=200 independent realizations). revision: yes
-
Referee: [Abstract] The augmentation of adversarial Q-learning to suppress stagnation-point trapping is described only qualitatively; no explicit form is given for the penalty term, reward shaping, or state-space modification (e.g., how volume penalization enters the Q-update or policy), which is load-bearing for assessing whether new failure modes arise away from the obstacle.
Authors: We have expanded the abstract to state the explicit modification: the reward function is augmented by an additive penalty term −α·|u·n|·χ_obst, where χ_obst is the volume-penalization mask, α=0.15, and the state vector is extended by the signed distance to the obstacle boundary. This term is applied only when the swimmer is within 0.2R of the obstacle; the Q-update itself remains unchanged. Full derivation appears in Sec. 3.2 of the manuscript. revision: yes
-
Referee: [Abstract] No results are shown comparing trajectories or performance statistics in the forward-cascade turbulent field far from the obstacle, leaving open whether the suppression heuristic produces globally improved navigation or merely trades one trapping mechanism for inefficient detours or instability.
Authors: Figures 4 and 5 already present ensemble statistics and representative trajectories both inside and outside the obstacle vicinity (r>3R). In the far field the SS still reduce mean travel time by 22% relative to NS while maintaining comparable path lengths, indicating that the local penalty does not induce global detours. We have added a one-sentence summary of these far-field metrics to the abstract. revision: yes
Circularity Check
Self-citation for base Q-learning; new obstacle augmentation tested via independent simulations
specific steps
-
self citation load bearing
[Abstract]
"We generalize recent Q-learning methods of J.K. Alageshan et al. [Phys.Rev.E 101, 043110 (2020)] and A. Gupta et al. [Physics of Fluids 37, 045107 (2025)] developed for non-interacting microswimmers that aim to move optimally from an initial position to a target, to account for the additional complication of an obstacle in the flow. [...] We augment our adversarial Q-learning Refs. [Alageshan_2020,Akanksha_2025] by suppressing the tendency of microswimmers to get trapped in stagnation points in the vicinity of the obstacle. We demonstrate that smart microswimmers (SS), which adopt our obstacle"
The base adversarial Q-learning algorithm and training setup are imported directly from self-cited prior works by overlapping authors (including A. Gupta), so the paper's performance claims for the augmented strategy rest on the un-rederived validity of that inherited framework even though the obstacle-specific suppression and new simulations are added.
full rationale
The paper is an empirical computational study that generalizes prior Q-learning methods to a new setting with an obstacle introduced via volume penalization. The central claim of outperformance by smart microswimmers is established through new simulation results comparing SS, NS, and SuS strategies rather than any mathematical derivation that reduces to its own inputs. The two self-citations supply the inherited base adversarial Q-learning framework, but this does not render the new augmentation or performance metrics circular because the results are generated and compared in the fresh obstacle-rich turbulent flow configuration.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Machine learning strategies for path-planning microswimmers in turbulent flows.Phys
Jaya Kumar Alageshan, Akhilesh Kumar Verma, J´ er´ emie Bec, and Rahul Pandit. Machine learning strategies for path-planning microswimmers in turbulent flows.Phys. Rev. E, 101:043110, Apr 2020
work page 2020
-
[2]
Rosanna A. Alegado and Nicole King. Bacterial influ- ences on animal origins.Cold Spring Harbor Perspectives in Biology, 6(11):a016162, 2014
work page 2014
-
[3]
Roberto Alonso-Matilla, Brato Chakrabarti, and David Saintillan. Transport and dispersion of active parti- cles in periodic porous media.Physical Review Fluids, 4(4):043101, 2019
work page 2019
-
[4]
Richard M. Berry. E. coli in motion.Physics Today, 58(2):64–65, 02 2005
work page 2005
-
[5]
L. Biferale, F. Bonaccorso, M. Buzzicotti, P. Clark Di Leoni, and K. Gustavsson. Zermelo’s problem: Op- timal point-to-point navigation in 2d turbulent flows us- ing reinforcement learning.Chaos: An Interdisciplinary Journal of Nonlinear Science, 29(10), October 2019
work page 2019
-
[6]
Guido Boffetta and Robert E. Ecke. Two-dimensional turbulence.Annual Review of Fluid Mechanics, 44(1):427–451, 2012
work page 2012
-
[7]
Chiara Calascibetta, Luca Biferale, Francesco Borra, An- tonio Celani, and Massimo Cencini. Taming lagrangian chaos with multi-objective reinforcement learning.The European Physical Journal E, 46(9), 2023
work page 2023
-
[8]
Priyam Chakraborty, Rahul Roy, and Shubhadeep Man- dal. Smart navigation of microswimmers in poiseuille flow via reinforcement learning.The European Physical Journal E, 48:30, 2025
work page 2025
-
[9]
Oleksandr Chepizhko and Fernando Peruani. Diffusion, subdiffusion, and trapping of active particles in heteroge- neous media.Physical Review Letters, 111:160604, 2013. 11
work page 2013
-
[10]
Flow navigation by smart microswim- mers via reinforcement learning.Phys
Simona Colabrese, Kristian Gustavsson, Antonio Celani, and Luca Biferale. Flow navigation by smart microswim- mers via reinforcement learning.Phys. Rev. Lett., 118:158004, Apr 2017
work page 2017
-
[11]
Simona Colabrese, Kristian Gustavsson, Antonio Celani, and Luca Biferale. Smart inertial particles.Phys. Rev. Fluids, 3:084301, Aug 2018
work page 2018
-
[12]
S. M. Cox and P. C. Matthews. Exponential time dif- ferencing for stiff systems.Journal of Computational Physics, 176(2):430–455, 2002
work page 2002
-
[13]
Abdallah Daddi-Moussa-Ider, Hartmut L¨ owen, and Benno Liebchen. Hydrodynamics can determine the op- timal route for microswimmer navigation.Communica- tions Physics, 4, 2021
work page 2021
-
[14]
Zakarya El Khiyati, Rapha¨ el Chesneaux, La¨ etitia Gi- raldi, and J´ er´ emie Bec. Steering undulatory microswim- mers using reinforcement learning.The European Physi- cal Journal E, 46(43), 2023
work page 2023
-
[15]
J Elgeti, R G Winkler, and G Gompper. Physics of microswimmers—single particle motion and collective behavior: a review.Reports on Progress in Physics, 78(5):056601, apr 2015
work page 2015
-
[16]
Haodong Feng, Dehan Yuan, Jiale Miao, Jie You, Yue Wang, Yi Zhu, and Dixia Fan. Efficient navigation of a robotic fish swimming across vortical flow fields.Journal of Hydrodynamics, 36:1118–1129
-
[17]
Uriel Frisch.Turbulence: The Legacy of A. N. Kol- mogorov. Cambridge University Press, Cambridge, 1995
work page 1995
-
[18]
Akanksha Gupta, Jaya Kumar Alageshan, Kiran Venkata Kolluru, and Rahul Pandit. Can flocking aid the path planning of microswimmers in turbulent flows?Physics of Fluids, 37(4):045107, 04 2025
work page 2025
-
[19]
Chater- jee, Shubhadeep Sadhukhan, Ravi Samtaney, and Ma- hendra K
Akanksha Gupta, Rohith Jayaram, Anando G. Chater- jee, Shubhadeep Sadhukhan, Ravi Samtaney, and Ma- hendra K. Verma. Energy and enstrophy spec- tra and fluxes for the inertial-dissipation range of two-dimensional turbulence.Physical Review E, 100(5):053101, November 2019
work page 2019
-
[20]
Targeted drug de- livery system: A review.Research Journal of Chemical Sciences, 1(2):135–138, 2011
Manish Gupta and Vimukta Sharma. Targeted drug de- livery system: A review.Research Journal of Chemical Sciences, 1(2):135–138, 2011
work page 2011
-
[21]
Gustavsson, Luca Biferale, Antonio Celani, and Si- mona Colabrese
K. Gustavsson, Luca Biferale, Antonio Celani, and Si- mona Colabrese. Finding efficient swimming strategies in a three-dimensional chaotic flow by reinforcement learn- ing.The European Physical Journal E, 40(110), 2017
work page 2017
-
[22]
M Y Hussaini and T A Zang. Spectral methods in fluid dynamics.Annual Review of Fluid Mechanics, 19(Volume 19, 1987):339–367, 1987
work page 1987
-
[23]
Benjamin Kadoch, Dmitry Kolomenskiy, Philippe Angot, and Kai Schneider. A volume penalization method for incompressible flows and scalar advection–diffusion with moving obstacles.Journal of Computational Physics, 231(12):4365–4383, 2012
work page 2012
-
[24]
Nicholas K.-R Kevlahan and Jean-Michel Ghidaglia. Computation of turbulent flow past an array of cylinders using a spectral method with brinkman penalization.Eu- ropean Journal of Mechanics - B/Fluids, 20(3):333–350, 2001
work page 2001
-
[25]
A. Kiverin, S. Luguev, and I. Yakovenko. Two-phase hydrodynamic model of active colloid motion, 2025
work page 2025
-
[26]
Christopher Koh, Laurent Pagnier, and Michael Chertkov. Physics-guided actor-critic reinforcement learning for swimming in turbulence.Physical Review Research, 7(1):013121, 2025
work page 2025
-
[27]
Dmitry Kolomenskiy and Kai Schneider. A fourier spec- tral method for the navier–stokes equations with volume penalization for moving solid obstacles.Journal of Com- putational Physics, 228(16):5687–5709, 2009
work page 2009
-
[28]
Revisiting the Master-Slave Architecture in Multi-Agent Deep Reinforcement Learning
Xiangyu Kong, Bo Xin, Fangchen Liu, and Yizhou Wang. Revisiting the master-slave architecture in multi- agent deep reinforcement learning.arXiv preprint arXiv:1712.07305, 2017
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[29]
Eric Lauga, W. R. DiLuzio, George M. Whitesides, and Howard A. Stone. Swimming in circles: Motion of bacte- ria near solid boundaries.Biophysical Journal, 90(2):400– 412, jan 2006
work page 2006
-
[30]
Springer, Dor- drecht, 4 edition, 2008
Marcel Lesieur.Turbulence in Fluids. Springer, Dor- drecht, 4 edition, 2008
work page 2008
-
[31]
Guanglai Li and Jay X. Tang. Accumulation of mi- croswimmers near a surface mediated by collision and ro- tational brownian motion.Phys. Rev. Lett., 103:078101, Aug 2009
work page 2009
-
[32]
Manzari, Yosi Shamay, Hiroto Kiguchi, Neal Rosen, Maurizio Scaltriti, and Daniel A
Mandana T. Manzari, Yosi Shamay, Hiroto Kiguchi, Neal Rosen, Maurizio Scaltriti, and Daniel A. Heller. Targeted drug delivery strategies for precision medicines.Nature Reviews Materials, 6(4):351–370, 2021
work page 2021
-
[33]
Selim Mecanna, Aurore Loisy, and Christophe Eloy. A critical assessment of reinforcement learning methods for microswimmer navigation in complex flows.The Euro- pean Physical Journal E, 48(10):58, 2025
work page 2025
-
[34]
Fran¸ cois-Ga¨ el Michalec, Sami Souissi, and Markus Holzner. Turbulence triggers vigorous swimming but hin- ders motion strategy in planktonic copepods.Journal of The Royal Society Interface, 12(106):20150158, 05 2015
work page 2015
-
[35]
Challenges and attempts to make intelligent microswimmers.Frontiers in Physics, 11:1279883, 2023
Chaojie Mo, Gaojin Li, and Xin Bian. Challenges and attempts to make intelligent microswimmers.Frontiers in Physics, 11:1279883, 2023
work page 2023
-
[36]
R´ emi Monthiller, Aurore Loisy, Mimi A. R. Koehl, Ben- jamin Favier, and Christophe Eloy. Surfing on turbu- lence: A strategy for planktonic navigation.Physical Review Letters, 129(6):064502, Aug 2022
work page 2022
-
[37]
Rupak Mukherjee, Akanksha Gupta, and Rajaraman Ganesh. Compressibility effects on quasistationary vor- tex and transient hole patterns through vortex merger. Physica Scripta, 94(11):115005, aug 2019
work page 2019
-
[38]
N. B. Padhan, D. Vincenzi, and R. Pandit. Interface- induced turbulence in viscous binary fluid mixtures, July
- [39]
-
[40]
Nadia Bihari Padhan and Rahul Pandit. The cahn– hilliard–navier–stokes framework for multiphase fluid flows: Laminar, turbulent and active.Journal of Fluid Mechanics, 1010:P1, 2025
work page 2025
-
[41]
Rahul Pandit, Debarghya Banerjee, Akshay Bhatna- gar, Marc Brachet, Anupam Gupta, Dhrubaditya Mi- tra, Nairita Pal, Prasad Perlekar, Samriddhi Sankar Ray, Vishwanath Shukla, and Dario Vincenzi. An overview of the statistical properties of two-dimensional turbulence in fluids with particles, conducting fluids, fluids with polymer additives, binary-fluid mi...
work page 2017
-
[42]
T. J. Pedley and J. B. Kessler. Hydrodynamic phenom- ena in suspensions of swimming microorganisms.Annual Review of Fluid Mechanics, 24(1):313–358, 1992
work page 1992
-
[43]
Jingran Qiu, Navid Mousavi, Kristian Gustavsson, Chunxiao Xu, Bernhard Mehlig, and Lihao Zhao. Nav- igation of micro-swimmers in steady flow: The im- portance of symmetries.Journal of Fluid Mechanics, 932:A10, 2022. 12
work page 2022
-
[44]
Margam Ramprasad, Shubhadeep Mandal, and Pallab Sinha Mahapatra. Motion of a microswimmer in a lat- tice of obstacles: Effect of thermal fluctuations.Physical Review E, 111(6):065402, Jun 2025
work page 2025
-
[45]
E. Schneider and H. Stark. Optimal steering of a smart active particle.EPL (Europhysics Letters), 127(6):64003, November 2019
work page 2019
-
[46]
Kai Schneider and Marie Farge. Numerical simulation of the transient flow behaviour in tube bundles using a vol- ume penalization method.Journal of Fluids and Struc- tures, 20(4):555–566, 2005
work page 2005
-
[47]
Rajesh Singh and James W. Lillard Jr. Nanoparticle- based targeted drug delivery.Experimental and Molecu- lar Pathology, 86(3):215–223, 2009
work page 2009
-
[48]
Saverio E. Spagnolie and Eric Lauga. Hydrodynamics of self-propulsion near a boundary: Predictions and accu- racy of far-field approximations.Journal of Fluid Me- chanics, 700:105–147, 2012
work page 2012
-
[49]
Saverio E. Spagnolie, Guillermo R. Moreno-Flores, Denis Bartolo, and Eric Lauga. Geometric capture and escape of a microswimmer colliding with an obstacle.Soft Mat- ter, 11(17):3396–3411, 2015
work page 2015
-
[50]
S. S. Suarez and A. A. Pacey. Sperm transport in the female reproductive tract.Human Reproduction Update, 12(1):23–37, 11 2005
work page 2005
-
[51]
R. S. Sutton and A. G. Barto. Reinforcement learn- ing: An introduction.IEEE Transactions on Neural Net- works, 9(5):1054–1054, 1998
work page 1998
-
[52]
Spectral methods in fluid dynamics (book review).Mathematics of Computation, 57(196):876–878, 1991
Eitan Tadmor. Spectral methods in fluid dynamics (book review).Mathematics of Computation, 57(196):876–878, 1991
work page 1991
-
[53]
J. Tailleur and M. Cates. Statistical mechanics of inter- acting run-and-tumble bacteria.Physical Review Letters, 100(21):218103, 2008
work page 2008
-
[54]
Daisuke Takagi, J´ er´ emie Palacci, Adam B. Braunschweig, Michael J. Shelley, and Jun Zhang. Hydrodynamic cap- ture of microswimmers into sphere-bound orbits.Soft Matter, 10(11):1784–1789, 2013
work page 2013
-
[55]
Rong Tan, Xiong Yang, Haojian Lu, and Yajing Shen. One-step formation of polymorphous sperm-like mi- croswimmers by vortex turbulence-assisted microfluidics. Nature Communications, 15:4761, 2024
work page 2024
-
[56]
Verma, Abhishek Kumar, and Akanksha Gupta
Mahendra K. Verma, Abhishek Kumar, and Akanksha Gupta. Hydrodynamic turbulence: Sweeping effect and taylor’s hypothesis via correlation function.Transactions of the Indian National Academy of Engineering, 5:649– 662, 2020
work page 2020
-
[57]
Microswim- mers in patterned environments.Soft Matter, 7:8810– 8815, 2011
Giovanni Volpe, Ivo Buttinoni, Dominik Vogt, Hans- J¨ urgen K¨ ummerer, and Clemens Bechinger. Microswim- mers in patterned environments.Soft Matter, 7:8810– 8815, 2011
work page 2011
-
[58]
Tiantian Xu, Yanming Guan, Jia Liu, and Xinyu Wu. Image-based visual servoing of helical microswimmers for planar path following.IEEE Transactions on Automa- tion Science and Engineering, 17(1):325–333, 2020
work page 2020
-
[59]
Navigating micro- and nano-motors/swimmers with machine learning: Challenges and future directions
Jueyi Xue, Hamid Alinejad-Rokny, and Kang Liang. Navigating micro- and nano-motors/swimmers with machine learning: Challenges and future directions. ChemPhysMater, 3:273–283, 2024
work page 2024
-
[60]
Zonghao Zou, Yuexin Liu, Alan C. H. Tsang, Y.-N. Young, and On Shun Pak. Adaptive micro-locomotion in a dynamically changing environment via context de- tection.Communications in Nonlinear Science and Nu- merical Simulation, 128:107666, 2024
work page 2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.