Pith. sign in

REVIEW 3 major objections 6 minor 44 references

Navigation of a Three-Link Microswimmer via Deep Reinforcement Learning

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A model-free reinforcement learning agent can learn stroke patterns for a three-link microswimmer that achieve at least 80 percent of the speed and efficiency of gaits found by classical optimization, and can then use those strokes to…

desk verdict PPO learns near-optimal three-link swimmer gaits, but the 'at least 80%' claim needs seed-level statistics and a reproducible optimal baseline before it's load-bearing. read the letter →

arxiv 2506.00084 v1 pith:VGRZ35PO submitted 2025-05-30 cs.RO physics.flu-dyn

Yuyang Lai , Sina Heydari , On Shun Pak , Yi Man This is my paper
classification cs.ROphysics.flu-dyn
keywords three-linkswimmerlow-Reynolds-numberlocomotiondeepreinforcementlearninggaitdiscoverystrokepatternoptimizationmicroswimmernavigationresistiveforcetheoryproximalpolicy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that model-free reinforcement learning can solve both gait discovery and targeted navigation for a three-link microswimmer at low Reynolds number, without being given any hydrodynamics or a prescribed stroke. With a proximal-policy-optimization agent acting on continuous hinge angular velocities, a velocity-only reward yields a fast, rectangular stroke pattern, while adding an energy penalty yields a slower, rounder, more efficient one. Compared with the velocity-optimal and efficiency-optimal gaits from classical optimization, the learned strategies reach at least 80 percent of the optimal average speed and of the optimal efficiency. The same policy can chase a sequence of target points to trace a star-shaped trajectory and can pursue a diffusing target moving at up to the swimmer's own speed. If this holds, RL becomes a viable alternative to explicit stroke design for microswimmers in tasks where the optimal gait is not known in advance.

What carries the argument

The mechanism that carries the argument is the resistive-force-theory model of Eq. (2), in which each link's local force is proportional to its local velocity with drag coefficients set by link slenderness; zero-net-force and zero-net-torque balance then gives the linear system Eq. (6) that maps hinge angular velocities to body motion. On the learning side, the carrying object is a PPO actor-critic agent acting on continuous hinge angular velocities, with observation $\left(\cos\theta_d, \sin\theta_d, \alpha_1, \alpha_2\right)$ and reward functions Eqs. (7) and (11). The comparison object is the closed stroke curve in the $(\alpha_1, \alpha_2)$ phase plane, benchmarked against the velocity-optimal and efficiency-optimal gaits of Ref. [40]; the efficiency measure itself follows Lighthill's ratio of towing work to actual work. What the machinery shows is that reward design alone, without periodicity constraints or gradient-based gait search, separates the two regimes of fast and efficient swimming.

What would settle it

Train or evaluate the same RL reward functions with the hydrodynamic forces recomputed by a boundary-integral or slender-body method that includes inter-link hydrodynamic interactions; if the resulting optimal strokes differ substantially and the RFT-trained policies fall below 80 percent of the new optima, the quantitative claim is an artifact of the drag model. A complementary check is an experimental three-link swimmer: if the learned velocity-focused and energy-aware strokes, played on physical hinges, do not produce the predicted direction and speeds in a viscous fluid, the model-to-physics transfer fails.

Watch

Extended reading notes

Core claim

The central claim is that a deep RL agent, trained only on a scalar reward, discovers stroke patterns for a three-link swimmer that are close to optimal for two different objectives. In both the velocity-focused and energy-aware strategies, the swimmer first steers and then translates along the target direction, and the learned phase-space stroke converges to a closed loop whose shape depends on the reward: rectangular for velocity, rounded for efficiency. The steady horizontal speeds are about 0.01284 for the velocity-focused strategy and 0.01176 for the energy-aware strategy, while the swimming efficiencies converge to about 0.854 percent and 1.077 percent respectively. Against the classical velocity-optimal and efficiency-optimal gaits, the RL strategies are at least 80 percent as effective in average velocity and average efficiency, even though the RL agent was never constrained to a periodic gait or a fixed stroke duration. The same trained policies allow the swimmer to follow a star-shaped path and to capture a Brownian target moving at up to half its own maximum speed, while closely tracking a target moving at full speed.

Load-bearing premise

The load-bearing assumption is that the resistive-force-theory model with drag ratio $\gamma=1/2$ faithfully describes the fluid forces on the three links, so the stroke patterns learned against it are the strokes a real microswimmer should use.

Editorial extensions

If this is right

  • A single RL policy can both steer and translate: after training, the swimmer converges from arbitrary initial orientations to straight motion along the target direction, so no separate steering controller is needed.
  • Reward design is a practical control knob: increasing the energy penalty weight $c$ shifts the stroke from the rectangular velocity-optimal shape to the rounded efficiency-optimal shape, raising efficiency while keeping navigation successful for $c \le 3$.
  • The velocity-focused trained agent can execute composite trajectories by chasing a sequence of target points, tracing a star-shaped path with both wide and sharp turns.
  • The same agent can pursue a target undergoing directed motion plus Brownian diffusion; it captures targets moving at zero and at half its own maximum speed and keeps close track of a target moving at full speed.
  • Because the learned strokes are produced without imposing periodicity or a fixed stroke duration, the RL approach applies to dynamically changing navigation tasks where classical single-period optimization would need reformulation.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 80 percent floor applies to two particular reward functions; varying the energy penalty weight $c$ continuously would likely trace a Pareto frontier between speed and efficiency, so a designer could select a gait for a given mission rather than accept one of two fixed strategies.
  • Because the gaits are optimized against resistive force theory with drag ratio $\gamma=1/2$, retraining with a resolved hydrodynamics solver or with measured drag coefficients would show whether the stroke shapes, not just the RL framework, transfer to physical microswimmers.
  • The observation vector contains no absolute position, only relative orientation and joint angles; extending the state to include target position or a memory of past observations would let the policy handle occlusions and more complex pursuit games.
  • Adding links would naturally grow the action space; the same reward scheme could then produce multi-link turning gaits, which the three-link geometry cannot express, and would test whether the 80 percent benchmark holds for higher-dimensional stroke spaces.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper applies deep reinforcement learning (PPO) to a three-link Purcell swimmer at low Reynolds number, using resistive-force-theory hydrodynamics. Two reward schemes are compared: a Velocity-Focused Strategy (VFS, Eq. 7) that rewards displacement along a target direction, and an Energy-Aware Strategy (EAS, Eq. 11) that adds a mechanical work penalty. The authors show that both strategies converge to periodic stroke patterns that enable straight swimming after a steering stage, and they compare the converged gaits and their average swimming speed and efficiency with the velocity-optimal (VO) and efficiency-optimal (EO) strokes of Tam and Hosoi (Ref. [40]). They report that VFS achieves over 80% of the VO average velocity and EAS over 80% of the EO average efficiency. The paper also demonstrates navigation along a star-shaped trajectory and pursuit of diffusing and moving targets using the VFS policy.

Significance. If the central quantitative claim is robust, this is a useful demonstration that model-free RL can discover stroke patterns for a model microswimmer that closely approach classical optimization results while also supporting more complex navigation tasks. The strengths of the paper include the detailed derivation of the equations of motion in the Supplemental Material, the reproducible description of the PPO algorithm, and the explicit parameter sensitivity study in Fig. S4. The main limitation is that the headline 80% comparison rests on single training runs and an incompletely specified reproduction of the reference optimal strokes, so the quantitative claim is not yet fully established. The work is within the scope of the journal and of interest to the microswimmer and RL-for-control communities.

major comments (3)
  1. [§V, Fig. 5] The headline claim that the RL strategies are 'at least 80% as effective' as the Tam-Hosoi optimal solutions is based on a single trained policy with no random-seed statistics. PPO is a stochastic algorithm (Supplemental Eq. S17 uses Gaussian action noise and clipped surrogate objectives), so a single run can land at a favorable point. The quoted ratios (about 85% for VFS/VO speed and about 82% for EAS/EO efficiency) leave only a small margin above the 80% threshold. The authors should train several independent policies (e.g., 5–10 seeds) and report the distribution or worst-case of the ratios, or otherwise provide a lower bound that establishes the 80% claim.
  2. [§III.B, Supplemental Fig. S4] The EAS efficiency claim is evaluated at c = 3, which the authors themselves show is at the edge of a stability cliff: at c = 4 the stroke pattern loses symmetry and the navigation success rate drops markedly. The sensitivity of the average efficiency and of the ratio EAS/EO to c in the neighborhood of 3 is not quantified with trials or error bars. If training-to-training or c-neighborhood variability is comparable to the margin over the 80% threshold, the efficiency claim is not robust. The authors should provide a sensitivity analysis around c = 3 or select a parameter value with a larger margin and justify that choice.
  3. [§IV.B, Fig. 5] The reproduction of the Tam-Hosoi reference values (VO and EO) is not sufficiently specified. The manuscript states that VO and EO are 'reproduced from Ref. [40]' but gives no description of the Fourier truncation, discretization, numerical optimization, or averaging protocol used for the reproduction, and no code or data are provided. Because the 80% claim is a ratio whose denominator is the reproduced reference value, the quantitative comparison is not independently verifiable. The authors should supply the reproduction details and ideally the reference data or code.
minor comments (6)
  1. [Abstract] There is a typo in the abstract: 'showca se' should be 'showcase'.
  2. [§III.A] The state definition 'S ∈ (x1, θ1, θ2, θ3)' is slightly inconsistent notation because x1 is a two-dimensional coordinate vector; rewriting it as S = (x1, y1, θ1, θ2, θ3) would be clearer.
  3. [§IV.B, Fig. 4] In Fig. 4, the labels 'µ0 = 0', 'µ0 = π/3', 'µ0 = π/2' appear to use the symbol µ0 for the initial link orientation; this should be θ0 to match the text.
  4. [§IV.B, Fig. 5] The bar-chart ordering in Fig. 5(c,d) should be clearly aligned with the text description in §IV.B ('VO highest, then VFS, EAS, EO' for velocity; 'EO highest, then EAS, VFS, VO' for efficiency). Please check that the bar labels and colors are unambiguous.
  5. [§V] The concluding sentence 'the strategies developed by RL are at least 80% as effective as the optimal solutions' should be qualified with the specific conditions (here, b = 6, c = 3, single-seed training, RFT model, the considered swimmer geometry) to avoid over-generalization.
  6. [§IV.C, Fig. 7] The capture threshold of 0.001 is quite small relative to the swimmer length; a brief note on how this threshold was chosen and on the sensitivity of the reported capture/non-capture outcomes to this value would strengthen the moving-target results.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: RL reward objectives are explicit, and the 80% claim is benchmarked against an independent external optimization study.

full rationale

The derivation chain is self-contained against external benchmarks rather than circular. The RL reward functions (Eqs. 7 and 11) explicitly define the optimization objectives; reporting that the resulting policies move toward the target and, for EAS, consume less energy is reporting success on those objectives, not presenting the objective as an independent prediction. The paper's headline quantitative claim—'at least 80% as effective' as VO/EO—is benchmarked against Tam and Hosoi (Ref. [40]), an independent, non-overlapping optimization study, and the VO/EO stroke data are reproduced from that reference rather than fitted from the RL policies. The PPO implementation cites prior group works [22, 24] for adaptation, but the supplementary material gives full algorithmic detail and standard references (Schulman; Adam), so the citation is not load-bearing. The efficiency advantage of EAS over VFS is expected from the explicit energy penalty in Eq. 11 and the definition of efficiency in Eqs. 10/13; however, the paper does not disguise this as a discovered law, and the 80% comparison to EO is not forced by construction. Concerns about single-seed training, hand-tuned c=3, or reproduction of Tam-Hosoi are correctness/reproducibility issues, not circularity. Therefore no circular step is exhibited.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the RFT model and the two-dimensional, inertialess assumption. No new physical entities are introduced. The free parameters b, c, the steering threshold, and the smoothing window are hand-tuned to make the training behave well and to define the reported metrics; none of them are fitted to match the external VO/EO benchmarks.

free parameters (4)
  • Reward scale b = 6
    Scaling factor in the VFS and EAS reward functions (Eqs. 7 and 11). The authors report that b < 6 slows convergence and b > 6 causes numerical instability, so b = 6 is a hand-tuned compromise.
  • Energy penalty weight c = 3
    Weight on the energy penalty in the EAS reward (Eq. 11). The supplement shows c = 4 leads to asymmetric strokes and a marked decrease in success rate, making c = 3 a post hoc boundary choice.
  • Steering threshold = 2.5 degrees
    Threshold on the averaged orientation angle used to classify the steering and translation stages (Section IV.A), which determines the reported speed and efficiency values.
  • Smoothing window = 35 steps on each side
    One motion period is estimated as about 70 steps; the authors average positions from 35 steps before to 35 steps after each point to compute the smoothed paths and derived quantities.
assumptions (4)
  • domain assumption Resistive force theory (RFT) for hydrodynamic forces
    Eq. 2 assumes the local hydrodynamic force is proportional to the local velocity with constant drag coefficients. This neglects hydrodynamic interactions between links and out-of-plane effects. Invoked in Section II and the supplementary material.
  • domain assumption Slender-body drag ratio gamma = C_parallel / C_perp = 1/2
    Fixed in Section II for a slender swimmer (a << L). This ratio enters the dimensionless equations of motion and the computed speeds and efficiencies, but is not derived within the paper.
  • domain assumption Inertialess, quasi-steady low-Reynolds-number dynamics
    The total hydrodynamic force and torque are set to zero (Eq. 4), following standard low-Reynolds-number swimming theory. This is a physical assumption that the swimmer's inertia is negligible.
  • domain assumption Two-dimensional motion constraint
    The swimmer moves in a plane with coordinates (e1, e2) and no out-of-plane degrees of freedom. This is stated in Section II and used throughout the analysis.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Navigation of a Three-Link Microswimmer via Deep Reinforcement Learning." pith.science (2026). https://pith.science/paper/VGRZ35PO

@misc{pith2026250600084,
  author       = {Pith},
  title        = {Pith review of: Navigation of a Three-Link Microswimmer via Deep Reinforcement Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VGRZ35PO}},
  note         = {Machine review of arXiv:2506.00084}
}
read the original abstract

Motile microorganisms develop effective swimming gaits to adapt to complex biological environments. Translating this adaptability to smart microrobots presents significant challenges in motion planning and stroke design. In this work, we explore the use of reinforcement learning (RL) to develop stroke patterns for targeted navigation in a three-link swimmer model at low Reynolds numbers. Specifically, we design two RL-based strategies: one focusing on maximizing velocity (Velocity-Focused Strategy) and another balancing velocity with energy consumption (Energy-Aware Strategy). Our results demonstrate how the use of different reward functions influences the resulting stroke patterns developed via RL, which are compared with those obtained from traditional optimization methods. Furthermore, we showcase the capability of the RL-powered swimmer in adapting its stroke patterns in performing different navigation tasks, including tracing complex trajectories and pursuing moving targets. Taken together, this work highlights the potential of reinforcement learning as a versatile tool for designing efficient and adaptive microswimmers capable of sophisticated maneuvers in complex environments.

Figures

Figures reproduced from arXiv: 2506.00084 by the authors.

Figure 1
Figure 1. FIG. 1 [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. FIG. 2 [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. (b, d). Next, we calculate the slope angle θs of the smoothed path to determine the averaged orientation of the swimmer. To do this, we compute the finite differences between consecutive smoothed positions to obtain the local slope at each point. By analyzing θs, we can assess how effectively the swimmer is aligning its motion with the desired target direction, thereby establishing a criterion for convergence. In th… view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: FIG. 4 [PITH_FULL_IMAGE:figures/full_fig_p012_4.png]
Figure 5
Figure 5. Figure 5: FIG. 5 [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: FIG. 6 [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: FIG. 7 [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

44 extracted references · 42 canonical work pages

  1. [40]

    Tam and A

    D. Tam and A. E. Hosoi, Optimal stroke patterns for purce ll’s three-link swimmer, Physical Review Letters 98, 068105 (2007)

    work page 2007

  2. [1]

    Lauga and T

    E. Lauga and T. R. Powers, The hydrodynamics of swimming m icroorganisms, Reports on progress in physics 72, 096601 (2009)

    work page 2009

  3. [2]

    Elgeti, R

    J. Elgeti, R. G. Winkler, and G. Gompper, Physics of micro swimmers—single particle motion and collective behavior: a review, Reports on progress in ph ysics 78, 056601 (2015)

    work page 2015

  4. [3]

    J. M. Yeomans, D. O. Pushkin, and H. Shum, An introduction to the hydrodynamics of swimming microorganisms, The European Physical Journal Sp ecial Topics 223, 1771 (2014)

    work page 2014

  5. [4]

    L. J. Fauci and R. Dillon, Biofluidmechanics of reproduct ion, Annu. Rev. Fluid Mech. 38, 371 (2006)

    work page 2006

  6. [5]

    Lauga, Bacterial hydrodynamics, Annual Review of Flu id Mechanics 48, 105 (2016)

    E. Lauga, Bacterial hydrodynamics, Annual Review of Flu id Mechanics 48, 105 (2016)

    work page 2016

  7. [6]

    P´ arducz, Ciliary movement and coordination in cilia tes, International review of cytology 21, 91 (1967)

    B. P´ arducz, Ciliary movement and coordination in cilia tes, International review of cytology 21, 91 (1967)

    work page 1967

  8. [7]

    W. Gao, D. Kagan, O. S. Pak, C. Clawson, S. Campuzano, E. Ch uluun-Erdene, E. Shipton, E. E. Fullerton, L. Zhang, E. Lauga, et al. , Cargo-towing fuel-free magnetic nanoswimmers for targeted drug delivery, small 8, 460 (2012)

    work page 2012

Show all 44 references
  1. [8]

    Ceylan, I

    H. Ceylan, I. C. Yasa, O. Yasa, A. F. Tabak, J. Giltinan, an d M. Sitti, 3d-printed biodegradable microswimmer for theranostic cargo delivery and release, A CS nano 13, 3353 (2019)

    2019

  2. [9]

    Zhang, J

    L. Zhang, J. J. Abbott, L. Dong, K. E. Peyer, B. E. Kratochv il, H. Zhang, C. Bergeles, and B. J. Nelson, Characterizing the swimming properties of artificial bacterial flagella, Nano letters 9, 3663 (2009). 19

    2009

  3. [10]

    Grosjean, G

    G. Grosjean, G. Lagubeau, A. Darras, M. Hubert, G. Lumay , and N. Vandewalle, Remote control of self-assembled microswimmers, Scientific repor ts 5, 16035 (2015)

    2015

  4. [11]

    U. K. Cheang and M. J. Kim, Self-assembly of robotic micr o-and nanoswimmers using mag- netic nanoparticles, Journal of Nanoparticle Research 17, 1 (2015)

    2015

  5. [12]

    Huang, M

    T.-Y. Huang, M. S. Sakar, A. Mao, A. J. Petruska, F. Qiu, X .-B. Chen, S. Kennedy, D. Mooney, and B. J. Nelson, 3d printed microtransporters: Compound mi cromachines for spatiotempo- rally controlled delivery of therapeutic agents, Advanced materials (Deerfield Beach, Fla.) 27,...

    2015

  6. [13]

    W. Hu, G. Z. Lum, M. Mastrangeli, and M. Sitti, Small-sca le soft-bodied robot with multi- modal locomotion, Nature 554, 81 (2018)

    2018

  7. [14]

    C. Ohm, M. Brehmer, and R. Zentel, Liquid crystalline el astomers as actuators and sensors, Advanced materials 22, 3366 (2010)

    2010

  8. [15]

    B. Dai, J. Wang, Z. Xiong, X. Zhan, W. Dai, C.-C. Li, S.-P. Feng, and J. Tang, Programmable artificial phototactic microswimmer, Nature nanotechnolo gy 11, 1087 (2016)

    2016

  9. [16]

    Palagi, Soft microrobots based on photoresponsive m aterials, Mechanically Responsive Ma- terials for Soft Robotics , 327 (2020)

    S. Palagi, Soft microrobots based on photoresponsive m aterials, Mechanically Responsive Ma- terials for Soft Robotics , 327 (2020)

    2020

  10. [17]

    Von Rohr, S

    A. Von Rohr, S. Trimpe, A. Marco, P. Fischer, and S. Palag i, Gait learning for soft microrobots controlled by light fields, in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE, 2018) pp. 6199–6206

    2018

  11. [18]

    Nassif, S

    X. Nassif, S. Bourdoulous, E. Eug` ene, and P.-O. Courau d, How do extracellular pathogens cross the blood–brain barrier?, Trends in microbiology 10, 227 (2002)

    2002

  12. [19]

    J. P. Celli, B. S. Turner, N. H. Afdhal, S. Keates, I. Ghir an, C. P. Kelly, R. H. Ewoldt, G. H. McKinley, P. So, S. Erramilli, et al. , Helicobacter pylori moves through mucus by reducing mucin viscoelasticity, Proceedings of the National Academ y of Sciences 106, 14321 (2009)

    2009

  13. [20]

    S. A. Mirbagheri and H. C. Fu, Helicobacter pylori coupl es motility and diffusion to actively create a heterogeneous complex medium in gastric mucus, Phy sical review letters 116, 198101 (2016)

    2016

  14. [21]

    A. C. H. Tsang, P. W. Tong, S. Nallan, and O. S. Pak, Self-l earning how to swim at low reynolds number, Physical Review Fluids 5, 074101 (2020)

    2020

  15. [22]

    Y. Jiao, F. Ling, S. Heydari, N. Heess, J. Merel, and E. Ka nso, Learning to swim in potential flow, Physical Review Fluids 6, 050505 (2021). 20

    2021

  16. [23]

    J. Qiu, N. Mousavi, K. Gustavsson, C. Xu, B. Mehlig, and L . Zhao, Navigation of micro- swimmers in steady flow: The importance of symmetries, Journ al of Fluid Mechanics 932, A10 (2022)

    2022

  17. [24]

    Z. Zou, Y. Liu, Y.-N. Young, O. S. Pak, and A. C. Tsang, Gai t switching and targeted navigation of microswimmers via deep reinforcement learni ng, Communications Physics 5, 158 (2022)

    2022

  18. [25]

    Zhu, W.-Z

    G. Zhu, W.-Z. Fang, and L. Zhu, Optimizing low-reynolds -number predation via optimal control and reinforcement learning, Journal of Fluid Mecha nics 944, A3 (2022)

    2022

  19. [26]

    K. Qin, Z. Zou, L. Zhu, and O. S. Pak, Reinforcement learn ing of a multi-link swimmer at low reynolds numbers, Physics of Fluids 35 (2023)

    2023

  20. [27]

    Colabrese, K

    S. Colabrese, K. Gustavsson, A. Celani, and L. Biferale , Flow navigation by smart microswim- mers via reinforcement learning, Physical review letters 118, 158004 (2017)

    2017

  21. [28]

    J. K. Alageshan, A. K. Verma, J. Bec, and R. Pandit, Machi ne learning strategies for path- planning microswimmers in turbulent flows, Physical Review E 101, 043110 (2020)

    2020

  22. [29]

    Schneider and H

    E. Schneider and H. Stark, Optimal steering of a smart ac tive particle, Europhysics Letters 127, 64003 (2019)

    2019

  23. [30]

    Muinos-Landin, A

    S. Muinos-Landin, A. Fischer, V. Holubec, and F. Cichos , Reinforcement learning with arti- ficial microswimmers, Science Robotics 6, eabd9285 (2021)

    2021

  24. [31]

    Y. Yang, M. A. Bevan, and B. Li, Micro/nano motor navigat ion and localization via deep reinforcement learning, Advanced Theory and Simulations 3, 2000034 (2020)

    2020

  25. [32]

    L. Yang, J. Jiang, F. Ji, Y. Li, K.-L. Yung, A. Ferreira, a nd L. Zhang, Machine learning for micro-and nanorobots, Nature Machine Intelligence , 1 (202 4)

  26. [33]

    Y. Liu, Z. Zou, O. S. Pak, and A. C. Tsang, Learning to coop erate for low-reynolds-number swimming: a model problem for gait coordination, Scientific Reports 13, 9397 (2023)

    2023

  27. [34]

    Amoudruz and P

    L. Amoudruz and P. Koumoutsakos, Independent control a nd path planning of microswimmers with a uniform magnetic field, Advanced Intelligent Systems 4, 2100183 (2022)

    2022

  28. [35]

    M. R. Behrens and W. C. Ruder, Smart magnetic microrobot s learn to swim with deep reinforcement learning, Advanced Intelligent Systems 4, 2200023 (2022)

    2022

  29. [36]

    E. M. Purcell, Life at low reynolds number, American jou rnal of physics 45, 3 (1977)

    1977

  30. [37]

    S. J. Lighthill, Mathematical biofluiddynamics (SIAM, 1975). 21

    1975

  31. [38]

    The Supplemental Material also contains Refs

    See Supplemental Material at [URL will be inserted by pu blisher] for additional details on the dynamic model of the three-link swimmer; the PPO framewo rk used to train the swim- mer’s control policy; the impact of training parameters on t he swimmer’s performance; and supple...

  32. [39]

    E. M. Purcell, The efficiency of propulsion by a rotating fl agellum, Proceedings of the National Academy of Sciences 94, 11307 (1997)

    1997

  33. [43]

    These tables outline the specific steps of the PPO algorithm employed in this work, and we refer readers to classical references for fur ther details [3]

    Here, K represents the total number of epochs the Adam optimizer uses d uring the optimization process. These tables outline the specific steps of the PPO algorithm employed in this work, and we refer readers to classical references for fur ther details [3]. III. THE IMP ACT OF...

  34. [44]

    S. J. Lighthill, Mathematical biofluiddynamics (SIAM, 1975)

    1975

  35. [45]

    D. P. Kingma and J. Ba, Adam: A method for stochastic optim ization, arXiv preprint arXiv:1412.6980 (2014)

    2014 arXiv

  36. [46]

    Schulman, F

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Kl imov, Proximal policy optimization algorithms, arXiv preprint arXiv:1707.06347 (2017). 11

    2017 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.