Pith. sign in

REVIEW 2 major objections 6 minor 61 references

Optimal Navigation in Microfluidics via the Optimization of a Discrete Loss

T0 review · 2 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read By optimizing a discrete loss that couples dynamics to travel time, ODIL trains microfluidic control policies with one to three orders of magnitude fewer evaluations than reinforcement learning, and keeps working where RL fails.

desk verdict A solid model-based control paper whose central closed-loop/robustness claims outrun the main benchmarks; worth a serious referee, but the authors should test multi-start policies or sharpen the wording. read the letter →

arxiv 2506.15902 v1 pith:NK2QA66J submitted 2025-06-18 physics.comp-ph cs.RO

Petr Karnakov , Lucas Amoudruz , Petros Koumoutsakos This is my paper · ORCID
classification physics.comp-phcs.RO
keywords microfluidicnavigationclosed-loopcontroldiscretelossoptimizationneuralnetworkpolicydirectcollocationreinforcementlearningcomparisonmagneticmicroswimmersvortextransport
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a method called ODIL—training a neural-network policy by minimizing a discrete loss built from the equations of motion plus the travel time—can solve time-optimal navigation problems for microscopic devices in viscous flows. The authors show on two main benchmarks, magnetic swimmers and passively transported particles in a vortex grid, that ODIL produces reliable closed-loop policies where a state-of-the-art reinforcement-learning baseline struggles, especially as the action and state dimensions grow. If the claims hold, ODIL gives a model-based alternative to RL for microfluidic control that needs one to three orders of magnitude fewer policy evaluations, which matters for applications like targeted drug delivery and environmental monitoring where the flow environment is complex and the controls are high-dimensional.

What carries the argument

The load-bearing object is the discrete loss $L(x,\theta)$: on a uniform grid of $N$ time points, the dynamics residual $\Delta x_{n+1/2}-f^{n+1/2}_\theta \Delta t$ is squared and summed, and the travel time enters through $\lambda T$ with $T=(N-1)\Delta t$. The step size $\Delta t$ is tied to the local dynamics by $\Delta t = \Delta x \cdot f_\theta / (f_\theta \cdot f_\theta)$, and the initial and final positions are imposed exactly as $x_0=x_{\text{start}}$ and $x_{N-1}=x_{\text{target}}$. The trajectory and the policy weights $\theta$ are optimized together by automatic differentiation and a standard gradient-based optimizer, aided by a multigrid decomposition that lets information propagate faster across the time grid. The policy itself is a fully connected network with two hidden layers of 128 neurons, tanh activations, and a case-specific output layer. This is direct collocation—trajectory optimization that treats discretized dynamics as constraints to be satisfied by optimization—with a neural network in place of a table of controls, turning the constrained optimal-control problem into an unconstrained one that standard gradient optimizers can solve.

What would settle it

Take the ODIL policy trained for the three-vortex case and evaluate it from one thousand starting positions sampled away from the single training initial condition, with small random perturbations added at each time step; if a substantial fraction of the particles fail to reach their targets while a reinforcement-learning policy with the same noise still succeeds, the paper's closed-loop and robustness claims would not survive.

Watch

Extended reading notes

Core claim

The central claim is that minimizing a single discrete loss with respect to both the discretized trajectory and the weights of a neural-network policy yields time-optimal, closed-loop navigation policies for microfluidic systems. The method converts the constrained problem of minimizing travel time subject to the ODE $\dot{x}=f(x,a_\theta(x))$ and fixed endpoints into the unconstrained minimization of $L(x,\theta)=\sum_{n=0}^{N-2}\|\Delta x_{n+1/2}-f^{n+1/2}_\theta \Delta t\|^2 + \lambda T$, where the dynamics are enforced by a midpoint-rule residual and $\lambda T$ penalizes slow arrival. Because the gradients of this loss flow through the governing equations, a small change in any control is felt along the whole trajectory; the paper argues this is why ODIL scales to problems where RL, which updates from sampled scalar rewards, fails. In the main benchmarks ODIL uses about $1.29\times 10^6$ policy evaluations in every case while RL uses on the order of $10^8$–$10^9$, and ODIL reaches the target more closely and succeeds where RL does not, such as transporting more than three particles through a vortex grid.

Load-bearing premise

The load-bearing premise is that optimizing the loss on a single trajectory from one initial condition produces a state-feedback policy that also works from other starting positions and under perturbations; the main comparisons do not test this directly, and the only supporting evidence sits in the supplementary material.

Editorial extensions

If this is right

  • ODIL needs about $1.29\times 10^6$ policy evaluations to converge in all main cases, while the RL baseline needs $10^7$–$10^9$; wall-clock times are lower by factors of 10–100.
  • For the vortex-guidance task, RL has a 100% success rate for two particles, 30% for three, and no valid policy at all for four or more, while ODIL succeeds for all tested counts up to eight.
  • For magnetic swimmers, ODIL drives the swimmers closer to the targets than the RL tolerance $\delta=0.1$ and converges in about an order of magnitude fewer epochs.
  • In the high-dimensional obstacle benchmark in the supplementary material, ODIL keeps near-optimal travel times up to dimension 20, whereas RL fails in more than half its runs at dimensions 16 and above.
  • ODIL requires a differentiable ODE model, so its advantage is conditional on having such a model; the paper indicates that unknown dynamics could be handled by combining ODIL with a data-driven model.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • An editorial extension: because the headline cost comparison trains on a single trajectory, the 10–100× wall-time advantage is strictly about one-trajectory training; switching to the multi-trajectory loss used in the supplementary material would multiply the ODIL cost by the number of trajectories, and the comparison would need to be rerun.
  • Another step beyond the paper: the discrete-loss mechanism is not tied to ODEs, so any system with a differentiable discrete residual—such as a PDE discretization of unsteady Stokes flow—could use the same joint optimization, a natural next step the paper only names as future work.
  • If gradient information from the dynamics is the real source of the advantage, ODIL's edge over RL should grow monotonically with action-space dimension; sweeping the vortex benchmark beyond eight vortices would directly test that mechanism.
  • Because the ODIL policy is only as good as its differentiable model, coupling the discrete loss with a learned residual that corrects model error would let the speed advantage survive under imperfect dynamics, a possibility the paper mentions but does not explore.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper introduces ODIL (Optimization of a Discrete Loss) for closed-loop navigation and control of microdevices in microfluidic flows. The method represents the control policy as a neural network and minimizes a discrete loss that combines an ODE residual (midpoint collocation) with a travel-time objective, using automatic differentiation, Adam, and a multigrid decomposition for acceleration. The authors benchmark ODIL against the reinforcement learning algorithm V-RACER on several problems: magnetic artificial bacterial flagella, passive particle transport in vortex lattices, and, in the supplementary material, a brachistochrone, path planning in shear and vortical background flows, a three-bead swimmer, and obstacle-avoidance in high-dimensional spaces. The main claims are that ODIL is more robust than RL, requires one to three orders of magnitude fewer policy evaluations, and remains reliable in high-dimensional action/state spaces where RL fails.

Significance. If the claims hold, ODIL is a practically useful model-based alternative to RL for microfluidic navigation, particularly in problems with known differentiable dynamics and high-dimensional control spaces. The paper has clear strengths: it validates the method on multiple benchmarks, reports medians with percentile bands over 10 (or 20) random seeds, and checks against analytical solutions in the brachistochrone and simple-shear cases. The use of a NN policy within a direct-collocation framework is a sensible and novel combination, and the detailed hyperparameter reporting in the supplementary material supports reproducibility. The central methodological idea is sound, but the breadth of the 'closed-loop' and 'robustness' claims is currently ahead of the evidence, especially for the high-dimensional benchmarks highlighted in the abstract.

major comments (2)
  1. [Abstract; Section 7; Sections S4/S5 and S6.3] The abstract describes ODIL as a 'closed-loop control method' that is 'more robust' than RL, but the main benchmarks (ABF swimmers and vortex transport) train the ODIL policy on a single trajectory from a single initial condition. The only test with multiple starting positions and stochastic dynamics is in S6.3, which considers a different, low-dimensional 2D shear-flow problem and trains on S=1000 starting points simultaneously via Eq. (S12). The ABF and vortex policies are never evaluated from off-training initial conditions or under noise. The paper itself acknowledges the single-initial-condition limitation in Section 8, but the abstract and the 'more robust' wording are not qualified accordingly. This is load-bearing because the closed-loop property and the robustness advantage over RL are central advertised contributions. The authors should either provide multi-start and stochastic-dynamics evaluations for the ABF and vortex benchmarks, or explicitly restrict the closed-loop/robustness claims to the cases in which they are tested.
  2. [Section 7; Table 1; S6.1] The quantitative speedup claim ('up to three orders faster') is based on a comparison with a single RL algorithm, V-RACER with REFER, and the RL reward design had to be customized substantially for individual problems (in S6.1 the generic reward formulation failed and a special reward with several constants was needed). The wall-time and policy-evaluation numbers are also reported only for RL runs that succeeded. The headline speedup is therefore contingent on this particular baseline and cannot be read as a general statement about RL as a whole. The text should explicitly scope the speedup to the V-RACER baseline, and ideally include at least one additional modern RL baseline (e.g., PPO or SAC) for the main benchmarks to support the broader claim.
minor comments (6)
  1. [Section 5] In the equation for the magnetic swimmers, the range 'i= 2,...,M' appears to be a typo; from the context (starting positions and parameters for all swimmers) it should read 'i= 1,...,M'.
  2. [Section 3, Eq. (4)] The notation for the scalar product and the expression for ∆t are defined after Eq. (4) is introduced. Please move these definitions before the loss function so the reader can parse the equation on first reading.
  3. [Table 1] The header 'W all time' is a typo; it should be 'Wall time'.
  4. [Section 7] The wall-time comparison in Table 1 lacks information about the computing environment (CPU/GPU model, framework version). Please specify the hardware and software configuration so the wall-time numbers can be reproduced.
  5. [Section S6.4, Eq. (S17)] The notation '−λxN c' is undefined; presumably it denotes the final center-of-mass position multiplied by λ. Please clarify the notation.
  6. [Section S6.3] In the stochastic evaluation with D=0.01, the text states that 'All trajectories reach the target', but no quantitative statistics (success rate, distribution of arrival times, or failure criteria) are reported. Please include summary statistics for the noisy rollout.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the ODIL loss and benchmarks are self-contained and tested against external references.

full rationale

The central derivation is the unconstrained loss L(x,θ)=Σ||Δx_{n+1/2}−f^{n+1/2}_θ Δt||^2+λT (Eq. 4), which is stated in full and is a standard direct-collocation residual with a NN policy; it does not presuppose the travel times, trajectories, or success rates reported later. The paper's headline claims (robustness, speed, high-dimensional performance) are tested against external references: the brachistochrone and shear-flow examples have analytical solutions, the obstacle problem has an exact trajectory, and RL (V-RACER) is an independent baseline, not a fitted input. The self-citations [9] and [47] identify the prior ODIL formulation and multigrid acceleration, but the present paper re-derives the loss and measures speed-up empirically in Table 1 and Table S1; no load-bearing conclusion is imported solely from those citations. No parameter is fitted to a subset of data and then called a prediction, and no uniqueness theorem or ansatz is imported from prior work. The acknowledged limitation that the main benchmarks train on a single initial condition (Section 8) concerns off-trajectory generalization and robustness, which is a validity question rather than a circularity; the supplementary material (S6.3) addresses it with S=1000 initial conditions and stochastic noise. Therefore the derivation chain is self-contained and no circular step is present.

Assumptions & free parameters 7 free parameters · 6 assumptions · 0 invented entities

The ledger is dominated by method hyperparameters and modeling assumptions because the paper makes a performance claim for a numerical method rather than a physical law. The main ODIL hyperparameters (lambda, eta, N, noise, multigrid) are hand-chosen and affect the reported speedups and accuracy; the RL comparison rests on a single baseline configuration. No free physical constants are fitted to data in the sense of a prediction claim, and no new physical entities are introduced. The modeling assumptions about the ODEs and the NN architecture are the main external inputs the central claim depends on.

free parameters (7)
  • Loss penalty lambda = 0.02 (ABFs), 0.1 (vortices, brachistochrone, obstacle), 0.01 (shear, vortical), 2 (three-bead)
    Chosen by hand for each case; balances ODE residual against travel time and is decreased across optimization rounds. The three-bead result is reported as sensitive to its value.
  • Learning rate schedule = eta=0.005 or 0.001; multiplied by 0.1 to 0.5 after five rounds
    Adam hyperparameters chosen per case; no automated tuning is described, and the schedules differ across benchmarks.
  • Time grid size N = 129 for most cases; 1025 for three-bead swimmer
    Controls temporal resolution and optimization cost; no grid-convergence study is reported for the main benchmarks.
  • Exploration noise standard deviation = 1/sqrt(N) on policy output p, sampled once in the first round
    Ad hoc term to help exploration; it is not based on data and is absent in later rounds.
  • Swimmer parameters omega_c,i and b_i = omega_c,i=i, b_i=1/i
    Chosen arbitrarily in SI S4 and argued experimentally realizable; determines the difficulty of the magnetic swimmer benchmark and hence the comparison.
  • RL baseline hyperparameters = kappa=10, tau=0.1, delta=0.1, 10,000 to 50,000 episodes, batch size B=256
    Hand-chosen for the single RL algorithm (V-RACER); the reported ODIL speedup and robustness advantage depends on this configuration.
  • Obstacle repulsion coefficient = 100 in eq. (S19)
    Chosen to prevent penetration of the obstacle; affects trajectories in the supplementary obstacle benchmark.
assumptions (6)
  • domain assumption Minimizing the discrete loss in eq. (4) is a valid relaxation of the constrained optimal control problem in eqs. (1)-(3).
    The paper relies on direct collocation practice and on the decreasing lambda schedule to drive residuals to small values; no equivalence proof is given for nonconvex NN policies.
  • domain assumption The ODE models capture the essential dynamics and neglected interactions do not change the conclusions.
    Section 5 states that hydrodynamic and magnetic interactions between swimmers are neglected; Section 6 assumes passive advection by a prescribed vortex flow.
  • domain assumption A two-hidden-layer NN with 128 units per layer is expressive enough to represent near-optimal policies.
    No capacity or approximation analysis is given; all benchmark results use this fixed architecture.
  • ad hoc to paper Adam with the stated schedules and multigrid decomposition converges to a useful minimum of the nonconvex loss.
    Convergence is empirical; there is no optimality guarantee. The multigrid representation is introduced in SI S1 to accelerate this convergence.
  • standard math The midpoint discretization with N=129 and RK2 evaluation at dt/4 accurately represents the continuous dynamics.
    Standard numerical analysis, and the paper checks some cases against analytical solutions, but no systematic grid-convergence study is shown for the central benchmarks.
  • standard math The potential-based reward shaping in eq. (5) preserves the optimal policy of the RL baseline.
    Cited to Ng et al. [58]; it ensures the RL reward is a valid proxy for the time-minimization objective.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Optimal Navigation in Microfluidics via the Optimization of a Discrete Loss." pith.science (2026). https://pith.science/paper/NK2QA66J

@misc{pith2026250615902,
  author       = {Pith},
  title        = {Pith review of: Optimal Navigation in Microfluidics via the Optimization of a Discrete Loss},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NK2QA66J}},
  note         = {Machine review of arXiv:2506.15902}
}
read the original abstract

Optimal path planning and control of microscopic devices navigating in fluid environments is essential for applications ranging from targeted drug delivery to environmental monitoring. These tasks are challenging due to the complexity of microdevice-flow interactions. We introduce a closed-loop control method that optimizes a discrete loss (ODIL) in terms of dynamics and path objectives. In comparison with reinforcement learning, ODIL is more robust, up to three orders faster, and excels in high-dimensional action/state spaces, making it a powerful tool for navigating complex flow environments.

Figures

Figures reproduced from arXiv: 2506.15902 by the authors.

Figure 1
Figure 1. Magnetic swimmers. (A) Velocity response of three swimmers depending on the driving frequency. (B, C, D) Distance to target, travel time, and number of epochs until convergence versus the number of swimmers for ODIL and RL . Shades are the 20th to 80th percentiles, solid lines are the medians over 10 realizations. The dashed line shows the threshold distance to target δ. v(ω; bi , ωc,i)p, i = 2, . . . , M, where M i… view at source ↗
Figure 2
Figure 2. Trajectories of M = 2, 4, and 6 magnetic swimmers with starting positions and target using ODIL (A) and RL (B). velocity field is a superposition of M × M vortices u(x) = M X×M i=1 ωi uV (x − ci), with intensities ωi ∈ (−1, 1) and centers ci forming a uniform grid in [0.5, M − 0.5] × [0.5, M − 0.5], and each vortex produces a flow proportional to uV (x, y) = e −(x 2+y 2 )/0.72(−y, x). The task is to transport the pa… view at source ↗
Figure 3
Figure 3. Manipulation by vortices. Performance of ODIL [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Manipulation by vortices using ODIL for M = 2, 4, and 7. Each row corresponds to one trajectory at times t/T = 0, 0.5, and 1. Vorticity field with arrows showing the intensity and orientation of vortices. 6 [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 50 canonical work pages

  1. [1]

    Compu- tational oncology—mathematical modelling of drug regimens for precision medicine.Nature reviews Clinical oncology, 13(4):242–254, 2016

    Dominique Barbolosi, Joseph Ciccolini, Bruno Lacarelle, Fabrice Barl´ esi, and Nicolas Andr´ e. Compu- tational oncology—mathematical modelling of drug regimens for precision medicine.Nature reviews Clinical oncology, 13(4):242–254, 2016

    work page 2016

  2. [2]

    Cell manipulation in microfluidics.Biofabrication, 5(2):022001, 2013

    Hoyoung Yun, Kisoo Kim, and Won Gu Lee. Cell manipulation in microfluidics.Biofabrication, 5(2):022001, 2013

    work page 2013

  3. [3]

    Micro-manipulation using rotational fluid flows induced by remote magnetic micro-manipulators.Journal of Applied Physics, 112(6), 2012

    Zhou Ye, Eric Diller, and Metin Sitti. Micro-manipulation using rotational fluid flows induced by remote magnetic micro-manipulators.Journal of Applied Physics, 112(6), 2012

    2012

  4. [4]

    Control and transport of passive particles using self-organized spinning micro-disks.IEEE Robotics and Automation Letters, 7(2):2156–2161, 2022

    Franco N Pi˜ nan Basualdo, Gaurav Gardi, Wendong Wang, Sinan O Demir, Aude Bolopion, Micha¨ el Gau- thier, Pierre Lambert, and Metin Sitti. Control and transport of passive particles using self-organized spinning micro-disks.IEEE Robotics and Automation Letters, 7(2):2156–2161, 2022

    work page 2022

  5. [5]

    Optimal navigation strategies for active particles.Europhysics Letters, 127(3):34003, 2019

    Benno Liebchen and Hartmut L¨ owen. Optimal navigation strategies for active particles.Europhysics Letters, 127(3):34003, 2019

    work page 2019

  6. [6]

    Independent control and path planning of microswimmers with a uniform magnetic field.Advanced Intelligent Systems, 4(3):2100183, 2022

    Lucas Amoudruz and Petros Koumoutsakos. Independent control and path planning of microswimmers with a uniform magnetic field.Advanced Intelligent Systems, 4(3):2100183, 2022

    work page 2022

  7. [7]

    Recent advances in microswimmers for biomedical applications

    Ada-Ioana Bunea and Rafael Taboryski. Recent advances in microswimmers for biomedical applications. Micromachines, 11(12):1048, 2020

    work page 2020

  8. [8]

    Light-driven micro-and nanomotors for envi- ronmental remediation.Environmental Science: Nano, 4(8):1602–1616, 2017

    Muhammad Safdar, Juliane Simmchen, and Janne J¨ anis. Light-driven micro-and nanomotors for envi- ronmental remediation.Environmental Science: Nano, 4(8):1602–1616, 2017

    work page 2017

Show all 61 references
  1. [9]

    Solving inverse problems in physics by optimizing a discrete loss: Fast and accurate learning without neural networks.PNAS Nexus, page pgae005, 01 2024

    Petr Karnakov, Sergey Litvinov, and Petros Koumoutsakos. Solving inverse problems in physics by optimizing a discrete loss: Fast and accurate learning without neural networks.PNAS Nexus, page pgae005, 01 2024

    2024

  2. [10]

    An introduction to trajectory optimization: How to do your own direct collocation

    Matthew Kelly. An introduction to trajectory optimization: How to do your own direct collocation. SIAM Review, 59(4):849–904, 2017

    2017

  3. [11]

    Direct collocation methods for trajectory optimization in constrained robotic systems.IEEE Transactions on Robotics, 2022

    Ricard Bordalba, Tobias Schoels, Llu´ ıs Ros, Josep M Porta, and Moritz Diehl. Direct collocation methods for trajectory optimization in constrained robotic systems.IEEE Transactions on Robotics, 2022

    2022

  4. [12]

    SIAM, 2010

    John T Betts.Practical methods for optimal control and estimation using nonlinear programming. SIAM, 2010

    2010

  5. [13]

    Back-propagation neural networks for nonlinear self-tuning adaptive control.IEEE control systems Magazine, 10(3):44–48, 1990

    F-C Chen. Back-propagation neural networks for nonlinear self-tuning adaptive control.IEEE control systems Magazine, 10(3):44–48, 1990

    1990

  6. [14]

    Neural networks for control sys- tems—a survey.Automatica, 28(6):1083–1112, 1992

    Kenneth J Hunt, D Sbarbaro, R ˙Zbikowski, and Peter J Gawthrop. Neural networks for control sys- tems—a survey.Automatica, 28(6):1083–1112, 1992

    1992

  7. [15]

    Constrained neural networks for approx- imate nonlinear model predictive control

    Saket Adhau, Vihangkumar V Naik, and Sigurd Skogestad. Constrained neural networks for approx- imate nonlinear model predictive control. In2021 60th IEEE Conference on Decision and Control (CDC), pages 295–300. IEEE, 2021

    2021

  8. [16]

    Pontryagin differentiable programming: An end-to-end learning and control framework.Advances in Neural Information Processing Systems, 33:7979–7992, 2020

    Wanxin Jin, Zhaoran Wang, Zhuoran Yang, and Shaoshuai Mou. Pontryagin differentiable programming: An end-to-end learning and control framework.Advances in Neural Information Processing Systems, 33:7979–7992, 2020

    2020

  9. [17]

    Differentiable mpc for end-to-end planning and control.Advances in neural information processing systems, 31, 2018

    Brandon Amos, Ivan Jimenez, Jacob Sacks, Byron Boots, and J Zico Kolter. Differentiable mpc for end-to-end planning and control.Advances in neural information processing systems, 31, 2018. 8

    2018

  10. [18]

    Efficient representation and approximation of model predictive control laws via deep learning.IEEE Transactions on Cybernetics, 50(9):3866–3878, 2020

    Benjamin Karg and Sergio Lucia. Efficient representation and approximation of model predictive control laws via deep learning.IEEE Transactions on Cybernetics, 50(9):3866–3878, 2020

    2020

  11. [19]

    Automatic differentiation in machine learning: a survey.Journal of Marchine Learning Research, 18:1– 43, 2018

    Atilim Gunes Baydin, Barak A Pearlmutter, Alexey Andreyevich Radul, and Jeffrey Mark Siskind. Automatic differentiation in machine learning: a survey.Journal of Marchine Learning Research, 18:1– 43, 2018

    2018

  12. [20]

    A formulation of nonlinear model predictive control using automatic differentiation.Journal of Process Control, 15(8):851–858, 2005

    Yi Cao. A formulation of nonlinear model predictive control using automatic differentiation.Journal of Process Control, 15(8):851–858, 2005

    2005

  13. [21]

    Neural odes as feedback policies for nonlinear optimal control.arXiv preprint arXiv:2210.11245, 2022

    Ilya Orson Sandoval, Panagiotis Petsagkourakis, and Ehecatl Antonio del Rio-Chanona. Neural odes as feedback policies for nonlinear optimal control.arXiv preprint arXiv:2210.11245, 2022

    2022 arXiv

  14. [22]

    Neural ordinary differ- ential equations.Advances in neural information processing systems, 31, 2018

    Ricky TQ Chen, Yulia Rubanova, Jesse Bettencourt, and David K Duvenaud. Neural ordinary differ- ential equations.Advances in neural information processing systems, 31, 2018

    2018

  15. [23]

    Second-order neural ode optimizer.Ad- vances in Neural Information Processing Systems, 34:25267–25279, 2021

    Guan-Horng Liu, Tianrong Chen, and Evangelos Theodorou. Second-order neural ode optimizer.Ad- vances in Neural Information Processing Systems, 34:25267–25279, 2021

    2021

  16. [24]

    Data-driven optimal prediction with control.arXiv preprint arXiv:2406.01991, 2024

    Aleksandr Katrutsa, Ivan Oseledets, and Sergey Utyuzhnikov. Data-driven optimal prediction with control.arXiv preprint arXiv:2406.01991, 2024

    2024 arXiv

  17. [25]

    Recent advances in parameteridentification techniques for ode

    Hans Georg Bock. Recent advances in parameteridentification techniques for ode. InNumerical Treat- ment of Inverse Problems in Differential and Integral Equations: Proceedings of an International Work- shop, Heidelberg, Fed. Rep. of Germany, August 30—September 3, 1982, pages ...

    1982

  18. [26]

    Sampling-based algorithms for optimal motion planning.The international journal of robotics research, 30(7):846–894, 2011

    Sertac Karaman and Emilio Frazzoli. Sampling-based algorithms for optimal motion planning.The international journal of robotics research, 30(7):846–894, 2011

    2011

  19. [27]

    Sampling-based algorithms for optimal motion planning using closed-loop prediction

    Oktay Arslan, Karl Berntorp, and Panagiotis Tsiotras. Sampling-based algorithms for optimal motion planning using closed-loop prediction. In2017 IEEE international conference on robotics and automation (ICRA), pages 4991–4996. IEEE, 2017

    2017

  20. [28]

    An adaptive sampling algorithm with dynamic iterative probability adjustment incorporating positional information.Entropy, 26(6):451, 2024

    Yanbing Liu, Liping Chen, Yu Chen, and Jianwan Ding. An adaptive sampling algorithm with dynamic iterative probability adjustment incorporating positional information.Entropy, 26(6):451, 2024

    2024

  21. [29]

    MIT press, 2018

    Richard S Sutton and Andrew G Barto.Reinforcement learning: An introduction. MIT press, 2018

    2018

  22. [30]

    Deep reinforcement learning based mobile robot navigation: A review.Tsinghua Science and Technology, 26(5):674–691, 2021

    Kai Zhu and Tao Zhang. Deep reinforcement learning based mobile robot navigation: A review.Tsinghua Science and Technology, 26(5):674–691, 2021

    2021

  23. [31]

    Glider soaring via reinforcement learning in the field.Nature, 562(7726):236–239, 2018

    Gautam Reddy, Jerome Wong-Ng, Antonio Celani, Terrence J Sejnowski, and Massimo Vergassola. Glider soaring via reinforcement learning in the field.Nature, 562(7726):236–239, 2018

    2018

  24. [32]

    Efficient collective swimming by harness- ing vortices through deep reinforcement learning.Proceedings of the National Academy of Sciences, 115(23):5849–5854, 2018

    Siddhartha Verma, Guido Novati, and Petros Koumoutsakos. Efficient collective swimming by harness- ing vortices through deep reinforcement learning.Proceedings of the National Academy of Sciences, 115(23):5849–5854, 2018

    2018

  25. [33]

    Learning efficient navigation in vortical flow fields.Nature communications, 12(1):7143, 2021

    Peter Gunnarson, Ioannis Mandralis, Guido Novati, Petros Koumoutsakos, and John O Dabiri. Learning efficient navigation in vortical flow fields.Nature communications, 12(1):7143, 2021

    2021

  26. [34]

    Flow navigation by smart microswimmers via reinforcement learning.Physical review letters, 118(15):158004, 2017

    Simona Colabrese, Kristian Gustavsson, Antonio Celani, and Luca Biferale. Flow navigation by smart microswimmers via reinforcement learning.Physical review letters, 118(15):158004, 2017

    2017

  27. [35]

    Luca Biferale, Fabio Bonaccorso, Michele Buzzicotti, Patricio Clark Di Leoni, and Kristian Gustavs- son. Zermelo’s problem: optimal point-to-point navigation in 2D turbulent flows using reinforcement learning.Chaos: An Interdisciplinary Journal of Nonlinear Science, 29(10):103...

    2019

  28. [36]

    Reinforcement learning of optimal active particle navigation.New Journal of Physics, 24(7):073042, 2022

    Mahdi Nasiri and Benno Liebchen. Reinforcement learning of optimal active particle navigation.New Journal of Physics, 24(7):073042, 2022

    2022

  29. [37]

    Path planning of magnetic microswim- mers in high-fidelity simulations of capillaries with deep reinforcement learning.arXiv preprint arXiv:2404.02171, 2024

    Lucas Amoudruz, Sergey Litvinov, and Petros Koumoutsakos. Path planning of magnetic microswim- mers in high-fidelity simulations of capillaries with deep reinforcement learning.arXiv preprint arXiv:2404.02171, 2024

    2024 arXiv

  30. [38]

    Point-to-point navigation of a fish-like swimmer in a vortical flow with deep reinforcement learning.Frontiers in Physics, 10:870273, 2022

    Yi Zhu, Jian-Hua Pang, and Fang-Bao Tian. Point-to-point navigation of a fish-like swimmer in a vortical flow with deep reinforcement learning.Frontiers in Physics, 10:870273, 2022

    2022

  31. [39]

    Deep reinforcement learning-based automatic exploration for navigation in unknown environment.IEEE transactions on neural networks and learning systems, 31(6):2064–2076, 2019

    Haoran Li, Qichao Zhang, and Dongbin Zhao. Deep reinforcement learning-based automatic exploration for navigation in unknown environment.IEEE transactions on neural networks and learning systems, 31(6):2064–2076, 2019

    2019

  32. [40]

    Playing atari with deep reinforcement learning.arXiv preprint arXiv:1312.5602, 2013

    Volodymyr Mnih, Koray Kavukcuoglu, David Silver, Alex Graves, Ioannis Antonoglou, Daan Wierstra, and Martin Riedmiller. Playing atari with deep reinforcement learning.arXiv preprint arXiv:1312.5602, 2013

    2013 arXiv

  33. [41]

    Remember and forget for experience replay

    Guido Novati and Petros Koumoutsakos. Remember and forget for experience replay. InInternational Conference on Machine Learning, pages 4851–4860. PMLR, 2019

    2019

  34. [42]

    Model-based rein- forcement learning for closed-loop dynamic control of soft robotic manipulators.IEEE Transactions on Robotics, 35(1):124–134, 2018

    Thomas George Thuruthel, Egidio Falotico, Federico Renda, and Cecilia Laschi. Model-based rein- forcement learning for closed-loop dynamic control of soft robotic manipulators.IEEE Transactions on Robotics, 35(1):124–134, 2018

    2018

  35. [43]

    Reinforcement learning for robust trajectory design of inter- planetary missions.Journal of Guidance, Control, and Dynamics, 44(8):1440–1453, 2021

    Alessandro Zavoli and Lorenzo Federici. Reinforcement learning for robust trajectory design of inter- planetary missions.Journal of Guidance, Control, and Dynamics, 44(8):1440–1453, 2021

    2021

  36. [44]

    The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care.Nature medicine, 24(11):1716–1720, 2018

    Matthieu Komorowski, Leo A Celi, Omar Badawi, Anthony C Gordon, and A Aldo Faisal. The artificial intelligence clinician learns optimal treatment strategies for sepsis in intensive care.Nature medicine, 24(11):1716–1720, 2018

    2018

  37. [45]

    Learning to drive in a day

    Alex Kendall, Jeffrey Hawke, David Janz, Przemyslaw Mazur, Daniele Reda, John-Mark Allen, Vinh- Dieu Lam, Alex Bewley, and Amar Shah. Learning to drive in a day. In2019 international conference on robotics and automation (ICRA), pages 8248–8254. IEEE, 2019

    2019

  38. [46]

    Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980, 2014

    2014 arXiv

  39. [47]

    Flow reconstruction by multiresolution optimization of a discrete loss with automatic differentiation.The European Physical Journal E, 46(7):59, 2023

    Petr Karnakov, Sergey Litvinov, and Petros Koumoutsakos. Flow reconstruction by multiresolution optimization of a discrete loss with automatic differentiation.The European Physical Journal E, 46(7):59, 2023

    2023

  40. [48]

    Chemotaxis of an elastic flagellated microrobot.Physical Review E, 108(4):044408, 2023

    Chaojie Mo, Qingfei Fu, and Xin Bian. Chemotaxis of an elastic flagellated microrobot.Physical Review E, 108(4):044408, 2023

    2023

  41. [49]

    Challenges and attempts to make intelligent microswimmers

    Chaojie Mo, Gaojin Li, and Xin Bian. Challenges and attempts to make intelligent microswimmers. Frontiers in Physics, 11:1279883, 2023

    2023

  42. [50]

    Mart´ ın Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjun...

    2015

  43. [51]

    Compiling machine learning programs via high-level tracing.Systems for Machine Learning, 4(9), 2018

    Roy Frostig, Matthew James Johnson, and Chris Leary. Compiling machine learning programs via high-level tracing.Systems for Machine Learning, 4(9), 2018

    2018

  44. [52]

    Elsevier, 2000

    Ulrich Trottenberg, Cornelius W Oosterlee, and Anton Schuller.Multigrid. Elsevier, 2000

    2000

  45. [53]

    Sergio M Martin, Daniel W¨ alchli, Georgios Arampatzis, Athena E Economides, Petr Karnakov, and Petros Koumoutsakos. Korali: Efficient and scalable software framework for bayesian uncertainty quantification and stochastic optimization.Computer Methods in Applied Mechanics and ...

    2022

  46. [54]

    Fast magnetic micropropellers with random shapes.Nano letters, 15(10):7064–7070, 2015

    Peter J Vach, Peter Fratzl, Stefan Klumpp, and Damien Faivre. Fast magnetic micropropellers with random shapes.Nano letters, 15(10):7064–7070, 2015

    2015

  47. [55]

    Simple swimmer at low reynolds number: Three linked spheres

    Ali Najafi and Ramin Golestanian. Simple swimmer at low reynolds number: Three linked spheres. Physical Review E, 69(6):062901, 2004

    2004

  48. [56]

    Microswimmers learning chemo- taxis with genetic algorithms.Proceedings of the National Academy of Sciences, 118(19):e2019683118, 2021

    Benedikt Hartl, Maximilian H¨ ubl, Gerhard Kahl, and Andreas Z¨ ottl. Microswimmers learning chemo- taxis with genetic algorithms.Proceedings of the National Academy of Sciences, 118(19):e2019683118, 2021

    2021

  49. [57]

    Cem-gd: Cross- entropy method with gradient descent planner for model-based reinforcement learning.arXiv preprint arXiv:2112.07746, 2021

    Kevin Huang, Sahin Lale, Ugo Rosolia, Yuanyuan Shi, and Anima Anandkumar. Cem-gd: Cross- entropy method with gradient descent planner for model-based reinforcement learning.arXiv preprint arXiv:2112.07746, 2021

    2021 arXiv

  50. [58]

    Policy invariance under reward transformations: Theory and application to reward shaping

    Andrew Y Ng, Daishi Harada, and Stuart Russell. Policy invariance under reward transformations: Theory and application to reward shaping. InIcml, volume 99, pages 278–287. Citeseer, 1999

    1999

  51. [59]

    Artificial bacterial flagella: Fabrication and magnetic control.Applied Physics Letters, 94(6):064107, 2009

    Li Zhang, Jake J Abbott, Lixin Dong, Bradley E Kratochvil, Dominik Bell, and Bradley J Nelson. Artificial bacterial flagella: Fabrication and magnetic control.Applied Physics Letters, 94(6):064107, 2009

    2009

  52. [60]

    Chiral colloidal molecules and observation of the propeller effect.Journal of the American Chemical Society, 135(33):12353–12359, 2013

    Debora Schamel, Marcel Pfeifer, John G Gibbs, Bj¨ orn Miksch, Andrew G Mark, and Peer Fischer. Chiral colloidal molecules and observation of the propeller effect.Journal of the American Chemical Society, 135(33):12353–12359, 2013

    2013

  53. [61]

    Selecting for function: solution synthesis of magnetic nanopropellers.Nano letters, 13(11):5373–5378, 2013

    Peter J Vach, Nicolas Brun, Mathieu Bennet, Luca Bertinetti, Marc Widdrat, Jens Baumgartner, Stefan Klumpp, Peter Fratzl, and Damien Faivre. Selecting for function: solution synthesis of magnetic nanopropellers.Nano letters, 13(11):5373–5378, 2013. 11 Supplementary Information...

    2013

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.