Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

A review on Deep Reinforcement Learning for Fluid Mechanics

T0 review · 3 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Deep reinforcement learning can serve as a gradient-free substitute for classical adjoint methods in fluid-flow optimization, and the paper demonstrates this by finding the same optimal control-cylinder positions as an adjoint method at…

desk verdict Useful review plus a new DRL-vs-adjoint control-cylinder comparison; the review is worth reading, but the new experiment's headline claim is not yet established because the reward is a short-horizon drag proxy. read the letter →

arxiv 1908.04127 v2 pith:OFLD3EX7 submitted 2019-08-12 physics.comp-ph cs.LGphysics.flu-dyn

Paul Garnier , Jonathan Viquerat , Jean Rabault , Aurélien Larcher , Alexander Kuhnle , Elie Hachem This is my paper · ORCID
classification physics.comp-phcs.LGphysics.flu-dyn
keywords deepreinforcementlearningfluidmechanicsflowcontrolshapeoptimizationadjointmethoddragreductiontransferautoencoder
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Deep reinforcement learning has begun to be applied to fluid mechanics problems that are nonlinear and high-dimensional, and this paper reviews the state of that field. Its own contribution is a concrete comparison: a DRL agent is trained to position a small control cylinder so as to reduce the drag on a square cylinder in laminar flow. With a reward equal to the drag reduction, and with observations supplied through an autoencoder-compressed velocity field, the agent recovered the same optimal positions as a classical adjoint sensitivity method at Reynolds numbers 40 and 100, and produced coherent results at Re=10. The authors also show that transfer learning between Reynolds numbers and parallel multi-environment training substantially cut the learning time. They conclude that DRL is a viable gradient-free tool for both flow control and shape optimization, though high-Reynolds and high-dimensional action spaces remain untested.

What carries the argument

The argument is carried by a coupled training loop: a CFD solver supplies velocity fields, a convolutional autoencoder compresses each field from about 8000 to 70 features, and a policy network receives those features together with the last two cylinder positions and the Reynolds number, and outputs an action, either a small displacement or a direct position. The reward is the instantaneous drag reduction, $r_t = C_D^0 - C_D$, the same quantity the adjoint method differentiates. The two policy-gradient algorithms used, A3C and PPO, are standard DRL choices: A3C is an asynchronous actor-critic method and PPO is a trust-region-style method with clipped updates. Transfer learning, reusing agents trained at a lower Reynolds number as initializations for higher ones, is what turns a five-hour training run into ten-hour runs for four agents on a single CPU.

What would settle it

Repeat the same optimization with episodes long enough for the wake to reach a quasi-steady state, or with a reward based on time-averaged drag over many shedding periods. If the final cylinder positions no longer match the adjoint-based optimum, the short-horizon reward is not a faithful proxy; if they still match, the proxy is sound.

Watch

Extended reading notes

Core claim

On the test case of a square main cylinder with a movable smaller control cylinder in an incompressible laminar flow, the paper's finding is that deep reinforcement learning converges to the same optimum as the classical adjoint method. Two architectures are used: a sequential A3C agent that moves the cylinder in small steps over 17 time steps, and a direct PPO agent that chooses a position in one step, at Reynolds numbers 10, 40, and 100. Both share the same reward, the difference between the drag of the main cylinder alone and the total drag of the two cylinders, so a positive reward means net drag reduction. The agents' observations include the last two positions, the Reynolds number, and neural-network-encoded features of the velocity field. Comparing with the adjoint-based sensitivity results, the authors report the same optimal positions at Re=40 and Re=100, and coherent results at Re=10, which they read as evidence that DRL can perform optimization without gradient information.

Load-bearing premise

The reward is measured over very short episodes, 17 time steps for one set of agents and a single action for the other, so everything leans on the assumption that such short-horizon drag readings stand in for the steady-state drag that the adjoint method optimizes.

Editorial extensions

If this is right

  • DRL can serve as a gradient-free alternative to adjoint-based optimization in laminar flow problems where computing sensitivities is expensive or impossible.
  • Transfer learning lets an agent trained at one Reynolds number initialize training at another, reducing computational cost compared with starting from scratch.
  • Parallel multi-environment training scales nearly perfectly with the number of simulations, so wall-clock training time drops substantially.
  • The same reward function supports both sequential control and direct one-shot optimization, giving a unified interface for control and design.
  • Autoencoder-based state compression allows DRL to handle high-dimensional fluid fields with small policy networks, which is relevant to experimental or large-scale CFD settings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the short-horizon reward proxy holds, the same validation protocol, DRL against an adjoint baseline, could be carried to flows where adjoint gradients are hard to obtain, such as turbulent wakes; the episode length would then need to be checked carefully.
  • Because the reward gives only a scalar drag difference, the agent may have converged to the same local optimum as the adjoint method rather than the global one; starting the direct agent from several initial positions would tell.
  • The autoencoder's compression of the velocity field from 8000 to 70 dimensions suggests DRL could be driven by experimental flow images instead of simulated fields, a step the paper does not test.
  • At higher Reynolds numbers the drag reduction alone may be too noisy a signal, so a time-averaged or multi-objective reward could be necessary before DRL can match adjoint results outside laminar flow.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper is a review of deep reinforcement learning (DRL) applied to fluid mechanics. The first part introduces RL/DRL basics (MDPs, DQN, policy gradients, actor-critic methods, TRPO/PPO), and the second part surveys representative applications: fish swimming and collective swimming, fluid-rigid-body control, microfluidic flow shaping, and active flow control of a cylinder wake. The final application section (§3.7) contains an original experiment in which six DRL agents (A3C and PPO, with and without autoencoder-based state compression) are trained to position a small control cylinder near a square cylinder at Re = 10, 40, and 100. The authors report that the DRL-optimal positions match those obtained by the adjoint-based sensitivity analysis of Meliga et al. (2014), and use this as evidence that DRL can reproduce classical gradient-based optimization results without gradient information.

Significance. The review component is useful and generally accurate: it collects algorithm choices, reward functions, and CFD coupling strategies in one place and is written accessibly for newcomers. The code for §3.7 is made publicly available, which supports reproducibility. If the agreement with the adjoint benchmark is robustly established, the §3.7 result would be a valuable demonstration that DRL can match a classical gradient-based method on a laminar optimization task without using gradients, while also benefiting from transfer learning and parallel multi-environment training. However, the evidence currently supplied is insufficient to establish that specific claim, so the significance of the paper is conditional on additional verification.

major comments (3)
  1. [§3.7, Table 3] The central claim that "the same optimal positions were found" depends on the reward r_t = C_D^0 - C_D being a valid proxy for the steady-state drag objective optimized by the adjoint method in Meliga et al. (2014). According to Table 3, agents 1.1/2.1/3.1 act over 17 time steps and agents 1.2/2.2/3.2 take a single action, but the manuscript does not report the physical time step, the horizon in convective time units, or any check that the drag at the end of an episode coincides with the quasi-steady drag at the proposed position. Because the flow past a displaced cylinder relaxes over a vortex-convection time scale, a truncated-horizon reward can advantage positions whose early transient benefit disappears once the wake reorganizes; for the sequential agents the reward is also history-dependent because the cylinder moves at each step. To support the comparison, the authors should evaluate the learned positions with longer rollouts or steady-state drag calculations and show that the transient reward used in training has the same optimum as the steady drag.
  2. [§3.7, Figure 11] The agreement with Meliga et al. (2014) is asserted qualitatively. The paper gives no numerical coordinates of the DRL-optimal positions, no drag coefficients at those positions, no distance or tolerance criterion for "same", no number of independent training runs, and no error bars. Figure 11 simply shows six final configurations without overlaying the adjoint reference positions. This is a load-bearing omission for the paper's main new result; the authors should provide a quantitative comparison and define the notion of agreement, or downgrade the claim from "same optimal positions" to a qualitative demonstration.
  3. [§3.7, training convergence] No learning curves, reward traces, or convergence diagnostics are reported for any of the six agents, despite the text citing training times and transfer-learning speedups. Since DRL training is stochastic, the absence of multiple seeds and of a reward-convergence check means the reader cannot assess whether the positions in Figure 11 represent converged optima or single trajectories. At minimum, report the reward evolution and repeat the training with at least three seeds for one representative configuration.
minor comments (4)
  1. [Table 2 / References] The citation "American Institute of Aeronautics and Astronautics, ed. 2008 Morphing Airfoils with Four Morphing Parameters" is not a citable paper; the underlying conference contribution and authors should be identified.
  2. [§3.7, Table 3] The mapping between the two architectures and the agent labels is confusing: Table 3 lists agents 1.1/2.1/3.1 as "Control over 17 time steps" and agents 1.2/2.2/3.2 as "Direct optimization", while the text describes the PPO single-action architecture first and the A3C sequential architecture second. Please make the algorithm/architecture correspondence explicit.
  3. [§2.2.1 and throughout] There are frequent copyediting issues, including missing spaces between citations (e.g., "Goodfellow et al. (2017) Strang (2019) Bottou et al. (2018)"), inconsistent title capitalization in the bibliography, and "time station" in Table 1; a careful proofreading pass is needed.
  4. [§4] The closing claim that DRL methods "proved to be robust when exposed to possible numerical noise" is stronger than what the reviewed single-realization studies and the new experiment show; please qualify it.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: Section 3.7's DRL-vs-adjoint comparison uses an external drag-based benchmark, not a fitted or self-referential objective.

full rationale

The paper's only new quantitative result (Section 3.7) is the claim that DRL-trained agents find the same optimal control-cylinder positions as the adjoint-based study of Meliga et al. (2014). The training reward is r_t = C_D^0 - C_D, a direct CFD drag measurement; it is not defined in terms of Meliga's positions, nor is any parameter fitted to make the DRL optima coincide with the adjoint optima. The benchmark is external, and the objective (drag reduction) is independently shared by the reference method, so agreement is a meaningful finding rather than an identity. The 17-step/single-action episode length raises a legitimate validation question about whether the truncated reward has the same argmax as steady-state drag, but that is a correctness risk, not a circular reduction. Self-citations appear (Rabault et al. 2019; Rabault & Kuhnle 2019; Garnier & Viquerat 2019 GitHub), mainly for flow setup and library tooling; none of these supplies the compared optimal-position result. No equation is equivalent by construction to another, and no fitted input is renamed as a prediction. Therefore no circular step is found.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The new experimental section depends on a small number of hand-chosen settings (episode length, latent dimension) and on the CFD solver being accurate. The paper does not report hyperparameter sweeps, convergence diagnostics, or multiple seeds for the square-cylinder agents.

free parameters (2)
  • Episode length = 17 time steps (agents 1.1, 2.1, 3.1); single action (agents 1.2, 2.2, 3.2)
    The number of simulation steps in an episode is chosen by hand and determines whether the drag reward reflects steady-state or transient behavior. This affects the optimization objective and the comparison with the adjoint benchmark.
  • Autoencoder latent dimension = 70 (reduced from ~8000)
    The latent dimension of the autoencoder is chosen a priori; if it discards flow features relevant to drag, the policy may optimize on incomplete observations.
assumptions (3)
  • domain assumption The incompressible Navier-Stokes equations as discretized by FEniCS accurately represent the flow in the square-cylinder test case.
    The DRL rewards and final drag values are computed from this CFD solver; any systematic error in the solver shifts the optimum positions.
  • domain assumption The reward function rt = C0_D - C_D is a sufficient scalarization of the optimization objective.
    The agent maximizes cumulative reward, but the final comparison is on the position that minimizes drag; if the reward does not align with the objective over the episode horizon, the found position may be suboptimal.
  • domain assumption The DRL algorithms (PPO and A3C) as implemented in stable-baselines converge to near-optimal policies in the tested settings.
    The claim that DRL finds the same optima as the adjoint method assumes the training runs have converged; no convergence criteria or multiple seeds are reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A review on Deep Reinforcement Learning for Fluid Mechanics." pith.science (2026). https://pith.science/paper/OFLD3EX7

@misc{pith2026190804127,
  author       = {Pith},
  title        = {Pith review of: A review on Deep Reinforcement Learning for Fluid Mechanics},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/OFLD3EX7}},
  note         = {Machine review of arXiv:1908.04127}
}
read the original abstract

Deep reinforcement learning (DRL) has recently been adopted in a wide range of physics and engineering domains for its ability to solve decision-making problems that were previously out of reach due to a combination of non-linearity and high dimensionality. In the last few years, it has spread in the field of computational mechanics, and particularly in fluid dynamics, with recent applications in flow control and shape optimization. In this work, we conduct a detailed review of existing DRL applications to fluid mechanics problems. In addition, we present recent results that further illustrate the potential of DRL in Fluid Mechanics. The coupling methods used in each case are covered, detailing their advantages and limitations. Our review also focuses on the comparison with classical methods for optimal control and optimization. Finally, several test cases are described that illustrate recent progress made in this field. The goal of this publication is to provide an understanding of DRL capabilities along with state-of-the-art applications in fluid dynamics to researchers wishing to address new problems with these methods.

Figures

Figures reproduced from arXiv: 1908.04127 by the authors.

Figure 1
Figure 1. DRL agent and its environment  An episode is equivalent to one game;  A state consists of the summary of all pieces and their positions;  Possible actions correspond to moving a piece of the game;  The reward may simply be +1 if the agent wins, −1 if it loses and 0 in case of a tie or a non-terminal game state. Generally, the reward is a signal of how ‘good’ a board situation is, which helps the agent to learn d… view at source ↗
Figure 2
Figure 2. figure 2. An input vector [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 2
Figure 2. Representation of a single artificial neuron and a basic 3-layer neural net￾work. function applied at its output. The learning process in neural networks consists in adjusting all the biases and weights of the network in order to reduce the value of a well-chosen loss function that represents the quality of the network prediction. This update is usually performed by a stochastic gradient method, in which the gradien… view at source ↗
Figures from the paper (10 more)
Figure 3
Figure 3. Figure 3: Number of publications mentioning "Deep reinforcement learning" per year since 2010 (data from www.scholar.google.com). In blue, we indicate some of the most important algorithms or achievements of the field. 3 Applications In this section, several applications combini…
Figure 4
Figure 4. Figure 4: Leader and follower swimmer, reproduced from Novati et al. (2017), with the displacements ∆x and ∆y as well as the orientation θ between the leader and the follower. where L is the length of the fish, Tp is the tail-beat frequency, φ is a phase-difference, t is the tim…
Figure 5
Figure 5. Figure 5: Leader and follower swimmer, reproduced from Verma et al. (2018) [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Fluid jets to control rigid body, reproduced from Ma et al. (2018). 3.3 Fluid directed rigid body control using deep reinforcement learning Ma et al. (2018) The control of complex environments that include two fluid flows interacting with multiple rigid bodies (cf figu…
Figure 7
Figure 7. Figure 7: Targets flow and results obtained from DRL agent, reproduced from Lee et al. (2018). 3.4 Flow shape design for microfluidic devices using deep reinforcement learning Lee et al. (2018) In Lee et al. (2018), the authors explore the capabilities of DRL for microfluidic fl…
Figure 8
Figure 8. Figure 8: Comparison of the velocity magnitude without (top) and with (bottom) active flow control, reproduced from Rabault et al. (2019). A clear modification of the cylinder wake, similar to what would be obtained with boat-tailing, is visible. ANN as the controller. Typically…
Figure 9
Figure 9. Figure 9: Illustration of the acceleration of ANN training through DRL using a multi environment approach (reproduced from Rabault & Kuhnle (2019)). Both plots illustrate the convergence of the ANN learning as a function of UTC time, depending on the number of environments used …
Figure 10
Figure 10. Figure 10: Architecture for agents 1.1, 2.1 and 3.1. Two different architectures are proposed. The first one is a classical DRL architecture, similar to what is shown in figure 1, but with a difference that episodes were reduced to a single action. In that case, the agent direct…
Figure 11
Figure 11. Figure 11: figure 11 [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 11
Figure 11. Figure 11: Results obtained by the trained agents. The environment consists of a square of unit lateral size centered in (0, 0), immersed in a rectangular domain of dimension [−5, 20] × [−5, 5].  Gym (Brockman et al., 2016) for the DRL environment,  Stable-baselines (Hill et a…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Transfer learning-enhanced deep reinforcement learning for aerodynamic airfoil optimisation subject to structural constraints

    cs.LG 2025-05 conditional novelty 5.0 of 10

    Transfer learning from a NeuralFoil surrogate to XFoil cuts DRL airfoil-optimization training time by about 86% while keeping aerodynamic performance close to full XFoil training.

Reference graph

Works this paper leans on

62 extracted references · 50 canonical work pages · cited by 1 Pith paper

  1. [1]

    Transactions of the Canadian Society for Mechanical Engineering 26 , 373--388

    Ali , N & Behdinan , K 2003 Optimal geometrical design of aircraft using genetic algorithms . Transactions of the Canadian Society for Mechanical Engineering 26 , 373--388

    work page 2003

  2. [2]

    , Bletcha , J , Hake , A , Johansson , B , Kehlet , B , Logg , A , Richardson , C

    A ln s, M.S. , Bletcha , J , Hake , A , Johansson , B , Kehlet , B , Logg , A , Richardson , C. , Ring , J. , Rognes , M.E. & Wells , G.N. 2015 The FEniCS P roject V ersion 1.5 . Archive of Numerical Software 3

    work page 2015

  3. [3]

    2008 Morphing A irfoils with F our M orphing P arameters\/

    American Institute of Aeronautics and Astronautics , ed. 2008 Morphing A irfoils with F our M orphing P arameters\/

    work page 2008

  4. [4]

    CoRR abs/1607.07086 (2015), 1--17

    Bahdanau , Dzmitry , Brakel , Philemon , Xu , Kelvin , Goyal , Anirudh , Lowe , Ryan , Pineau , Joelle , Courville , Aaron & Bengio, Yoshua 2016 An A ctor- C ritic A lgorithm for S equence P rediction . CoRR abs/1607.07086 (2015), 1--17

    arXiv 2015

  5. [5]

    1957 A M arkovian D ecision P rocess

    Bellman , R. 1957 A M arkovian D ecision P rocess . Journal of Mathematics and Mechanics 6 (5), 679--684

    work page 1957

  6. [6]

    & Dreyfus , S

    Bellman , R. & Dreyfus , S. E. 1962 Applied dynamic programming\/ . Princeton University Press Princeton, N.J

    work page 1962

  7. [7]

    Physics of fluids 17 (9), 097101

    Bergmann, Michel , Cordier, Laurent & Brancher, Jean-Pierre 2005 Optimal rotary control of the cylinder wake using proper orthogonal decomposition reduced-order model . Physics of fluids 17 (9), 097101

    work page 2005

  8. [8]

    arXiv e-prints , arXiv:arXiv: 1812.03823

    Bewley , Alex , Rigley , Jessica , Liu , Yuxuan , Hawke , Jeffrey , Shen , Richard , Lam , Vinh-Dieu & Kendall , Alex 2018 Learning to D rive from S imulation without R eal W orld L abels . arXiv e-prints , arXiv:arXiv: 1812.03823

    arXiv 2018

Show all 62 references
  1. [9]

    & Nocedal , Jorge 2018 Optimization Methods for Large-Scale Machine Learning

    Bottou , L\'eon , Curtis , Frank E. & Nocedal , Jorge 2018 Optimization Methods for Large-Scale Machine Learning . SIAM Rev. 60 , 223--311

    2018

  2. [10]

    arXiv e-prints

    Brockman , Greg , Cheung , Vicki , Pettersson , Ludwig , Schneider , Jonas , Schulman , John , Tang , Jie & Zaremba , Wojciech 2016 OpenAI G ym . arXiv e-prints

    2016

  3. [11]

    Science , arXiv:arXiv: https://science.sciencemag.org/content/early/2019/07/10/science.aay2400

    Brown, Noam & Sandholm, Tuomas 2019 Superhuman AI for multiplayer poker . Science , arXiv:arXiv: https://science.sciencemag.org/content/early/2019/07/10/science.aay2400

    2019

  4. [12]

    Bucci , Michele Alessandro , Semeraro , Onofrio , Allauzen , Alexandre , Wisniewski , Guillaume , Cordier , Laurent & Mathelin , Lionel 2019 Control of chaotic systems by D eep R einforcement L earning . arXiv p. arXiv:1906.07672

    2019 arXiv

  5. [13]

    Physical Review Letters 118 (15), 158004

    Colabrese , Simona , Gustavsson , Kristian , Celani , Antonio & Biferale , Luca 2017 Flow N avigation by S mart M icroswimmers via R einforcement L earning . Physical Review Letters 118 (15), 158004

    2017

  6. [14]

    Foundations and trends in machine learning

    Fran c ois-lavet , Vincent , Henderson , Peter , Islam , Riashat & Bellemare , Marc G 2018 An I ntroduction to D eep R einforcement L earning . Foundations and trends in machine learning

    2018

  7. [15]

    https://github.com/DonsetPG/fenics-DRL

    Garnier , Paul & Viquerat , Jonathan 2019 Position of a control cylinder to reduce drag. https://github.com/DonsetPG/fenics-DRL

    2019

  8. [16]

    , Tchieu , A

    Gazzola , M. , Tchieu , A. A. , Alexeev , D. , de Brauer , A. & Koumoutsakos , P. 2016 Learning to school in the presence of hydrodynamic interactions . Journal of Fluid Mechanics 789 , 726--749

    2016

  9. [17]

    MIT Press

    Goodfellow , Ian , Bengio , Yoshua & Courville , Aaron 2017 The D eep L earning B ook \/ . MIT Press

    2017

  10. [18]

    Yousuff 2016 A statistical learning strategy for closed-loop control of fluid flows

    Gu \'e niat , Florimond , Mathelin , Lionel & Hussaini , M. Yousuff 2016 A statistical learning strategy for closed-loop control of fluid flows . Theoretical and Computational Fluid Dynamics 30 (6), 497--510

    2016

  11. [19]

    , Biferale , L

    Gustavsson , K. , Biferale , L. , Celani , A. & Colabrese , S. 2017 Finding E fficient S wimming S trategies in a T hree D imensional C haotic F low by R einforcement L earning . arXiv e-prints , arXiv:arXiv: 1711.05826

    2017 arXiv

  12. [20]

    https://github.com/hill-a/stable-baselines

    Hill, Ashley , Raffin, Antonin , Ernestus, Maximilian , Gleave, Adam , Traore, Rene , Dhariwal, Prafulla , Hesse, Christopher , Klimov, Oleg , Nichol, Alex , Plappert, Matthias , Radford, Alec , Schulman, John , Sidor, Szymon & Wu, Yuhuai 2018 Stable B aselines. https://github...

    2018

  13. [21]

    1735–1780

    Hochreiter , Sepp & Schmidhuber , Jürgen 1997 Long short-term memory . Neural Comput 9 , "1735–1780"

    1997

  14. [22]

    Hou Tsang , Alan Cheng , Tong , Pun Wai , Nallan , Shreyes & Pak , On Shun 2018 Self-learning how to swim at low R eynolds number . arXiv p. arXiv:1808.07639

    2018 arXiv

  15. [23]

    , Isola, Phillip , Stadie, Bradly C

    Houthooft, Rein , Chen, Richard Y. , Isola, Phillip , Stadie, Bradly C. , Wolski, Filip , Ho, Jonathan & Abbeel, Pieter 2018 Evolved Policy Gradients . arXiv e-prints

    2018

  16. [24]

    Howard , R. A. 1960 Dynamic programming and M arkov processes\/ . Technology Press and Wiley, New York

    1960

  17. [25]

    arXiv e-prints , arXiv:arXiv: 1807.00412

    Kendall , Alex , Hawke , Jeffrey , Janz , David , Mazur , Przemyslaw , Reda , Daniele , Allen , John-Mark , Lam , Vinh-Dieu , Bewley , Alex & Shah , Amar 2018 Learning to D rive in a D ay . arXiv e-prints , arXiv:arXiv: 1807.00412

    2018 arXiv

  18. [26]

    https://www.technologyreview.com/s/611902/google-just-gave-control-over-data-center-cooling-to-an-ai/

    Knight , Will 2018 Google just gave control over data center cooling to an AI . https://www.technologyreview.com/s/611902/google-just-gave-control-over-data-center-cooling-to-an-ai/

    2018

  19. [27]

    Journal of Aircraft 33 , 962--969

    Lee , J & Hajela , P 1996 Parallel genetic algorithm implementation in multidisciplinary rotor blade design . Journal of Aircraft 33 , 962--969

    1996

  20. [28]

    CoRR abs/1811.12444 , 1--10

    Lee , Xian Yeow , Balu , Aditya , Stoecklein , Daniel , Ganapathysubramanian , Baskar & Sarkar , Soumik 2018 Flow S hape D esign for M icrofluidic D evices U sing D eep R einforcement L earning . CoRR abs/1811.12444 , 1--10

    2018 arXiv

  21. [29]

    , Hunt, Jonathan J

    Lillicrap, Timothy P. , Hunt, Jonathan J. , Pritzel, Alexander , Heess, Nicolas , Erez, Tom , Tassa, Yuval , Silver, David & Wierstra, Daan 2015 Continuous control with deep reinforcement learning . arXiv e-prints

    2015

  22. [30]

    PhD thesis, Carnegie Mellon University, Pittsburgh, PA, USA, uMI Order No

    Lin , Long-Ji 1993 Reinforcement L earning for R obots U sing N eural N etworks . PhD thesis, Carnegie Mellon University, Pittsburgh, PA, USA, uMI Order No. GAX93-22750

    1993

  23. [31]

    Neurocomputing 139 , 84--96

    Liou, Cheng-Yuan , Cheng, Wei-Chen , Liou, Jiun-Wei & Liou, Daw-Ran 2014 Autoencoder for words . Neurocomputing 139 , 84--96

    2014

  24. [32]

    ACM Transactions on Graphics 37 , 1--11

    Ma, Pingchuan , Tian, Yunsheng , Pan, Zherong , Ren, Bo & Manocha, Dinesh 2018 Fluid directed rigid body control using deep reinforcement learning . ACM Transactions on Graphics 37 , 1--11

    2018

  25. [33]

    International Journal for Numerical Methods in Fluids 30 , 149--159

    M\"akinen , R.A.E , Periaux , J & Toivanen , J 1999 Multidisciplinary shape optimization in aerodynamics and electromagnetics using genetic algorithms . International Journal for Numerical Methods in Fluids 30 , 149--159

    1999

  26. [34]

    International Journal of Thermal Sciences 43 , 477--487

    Matos , R , Laursen , T , Vargas , J & Bejan , A 2004 Three-dimensional optimization of staggered finned circular and elliptic tubes in forced convection . International Journal of Thermal Sciences 43 , 477--487

    2004

  27. [35]

    Physics of Fluids 26 , 26,104101

    Meliga, Philippe , Boujo, Edouard , Pujals, Gregory & Gallaire, Fran c ois 2014 Sensitivity of aerodynamic forces in laminar and turbulent flow past a square cylinder . Physics of Fluids 26 , 26,104101

    2014

  28. [36]

    CoRR abs/1312.5602

    Mnih , Volodymyr , Kavukcuoglu , Koray , Silver , David , Graves , Alex , Antonoglou , Ioannis , Wierstra , Daan & Riedmiller , Martin 2013 Playing A tari with D eep R einforcement L earning . CoRR abs/1312.5602

    2013 arXiv

  29. [37]

    , Harley , Tim , Silver , David & Kavukcuoglu , Koray 2016 Asynchronous M ethods for D eep R einforcement L earning

    Mnih , Volodymyr , Puigdom \`e nech Badia , Adri \`a , Mirza , Mehdi , Graves , Alex , Lillicrap , Timothy P. , Harley , Tim , Silver , David & Kavukcuoglu , Koray 2016 Asynchronous M ethods for D eep R einforcement L earning . arXiv e-prints , arXiv:arXiv: 1602.01783

    2016 arXiv

  30. [38]

    Computers & Fluids 33 , 849--858

    Muyl , F , Dumas , L & Herbert , V 2004 Hybrid method for aerodynamic shape optimization in automotive industry . Computers & Fluids 33 , 849--858

    2004

  31. [39]

    & Koumoutsakos , Petros 2017 Synchronised swimming of two fish

    Novati , Guido , Verma , Siddhartha , Alexeev , Dmitry , Rossinelli , Diego , van Rees , Wim M. & Koumoutsakos , Petros 2017 Synchronised swimming of two fish . Bioinspiration and Biomimetics 12 (3), 036001

    2017

  32. [40]

    https://blog.openai.com/openai-five/

    OpenAI 2018 Open AI F ive. https://blog.openai.com/openai-five/

    2018

  33. [41]

    CoRR abs/1710.06542

    Pinto , Lerrel , Andrychowicz , Marcin , Welinder , Peter , Zaremba , Wojciech & Abbeel , Pieter 2017 Asymmetric A ctor C ritic for I mage- B ased R obot L earning . CoRR abs/1710.06542

    2017 arXiv

  34. [42]

    Qiu , Jingran , Zhao , Lihao , Xu , Chunxiao & Yao , Yichen 2018 Swimming strategy of settling elongated micro-swimmers by reinforcement learning . arXiv p. arXiv:1811.10880

    2018 arXiv

  35. [43]

    Journal of Fluid Mechanics 865 , 281--302

    Rabault , Jean , Kuchta , Miroslav , Jensen , Atle , R \'e glade , Ulysse & Cerardi , Nicolas 2019 Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control . Journal of Fluid Mechanics 865 , 281--302

    2019

  36. [44]

    arXiv e-prints , arXiv:arXiv: 1906.10382

    Rabault , Jean & Kuhnle , Alexander 2019 Accelerating D eep R einforcement L earning of A ctive F low C ontrol strategies through a multi-environment approach . arXiv e-prints , arXiv:arXiv: 1906.10382

    2019 arXiv

  37. [45]

    In Flow simulation with H igh- P erformance C omputers II \/ , pp

    Sch \"a fer, Michael , Turek, Stefan , Durst, Franz , Krause, Egon & Rannacher, Rolf 1996 Benchmark computations of laminar flow around a cylinder . In Flow simulation with H igh- P erformance C omputers II \/ , pp. 547--566 . Springer

    1996

  38. [46]

    arXiv e-prints , arXiv:arXiv: 1511.05952

    Schaul , Tom , Quan , John , Antonoglou , Ioannis & Silver , David 2015 Prioritized E xperience R eplay . arXiv e-prints , arXiv:arXiv: 1511.05952

    2015 arXiv

  39. [47]

    & Abbeel , Pieter 2015 Trust R egion P olicy O ptimization

    Schulman , John , Levine , Sergey , Moritz , Philipp , Jordan , Michael I. & Abbeel , Pieter 2015 Trust R egion P olicy O ptimization . arXiv e-prints , arXiv:arXiv: 1502.05477

    2015 arXiv

  40. [48]

    arXiv e-prints pp

    Schulman, John , Moritz, Philipp , Levine, Sergey , Jordan, Michael & Abbeel, Pieter 2016 High-Dimensional Continuous Control Using Generalized Advantage Estimation . arXiv e-prints pp. 1--14

    2016

  41. [49]

    & Sontag, E.D

    Siegelmann, H.T. & Sontag, E.D. 1995 On the C omputational P ower of N eural N ets . Journal of Computer and System Sciences 50 (1), 132--150

    1995

  42. [50]

    Nature 550

    Silver , David , Schrittwieser , Julian , Simonyan , Karen , Antonoglou , Ioannis , Huang , Aja , Guez , Arthur , Hubert , Thomas , Baker , Lucas , Lai , Matthew , Bolton , Adrian , Chen , Yutian , Lillicrap , Timothy , Hui , Fan , Sifre , Laurent , van den Driessche , George ...

    2017

  43. [51]

    Wellesley-Cambridge Press

    Strang , Gilbert 2019 Linear Algebra and Learning from Data \/ . Wellesley-Cambridge Press

    2019

  44. [52]

    Machine Learning 3 (1), 9--44

    Sutton , R.S 1988 Learning to P redict by the M ethod of T emporal D ifferences . Machine Learning 3 (1), 9--44

    1988

  45. [53]

    Sutton , R. S. & Barto , A. G. 1998 Reinforcement L earning: A n I ntroduction\/ . MIT Press, Cambridge, MA

    1998

  46. [54]

    Sutton , R. S. & Barto , A. G. 2018 Reinforcement L earning: A n I ntroduction\/ . MIT Press, Cambridge, MA

    2018

  47. [55]

    Communications of the ACM 38

    Tesauro , Gerald 1995 Temporal D ifference L earning and TD - G ammon . Communications of the ACM 38

    1995

  48. [56]

    IEEE Transactions p

    Tsitsiklis , John & Van Roy , Benjamin 1997 An analysis of temporal-difference learning with function approximation . IEEE Transactions p. 674–690

    1997

  49. [57]

    arXiv e-prints , arXiv:arXiv: 1509.06461

    van Hasselt , Hado , Guez , Arthur & Silver , David 2015 Deep R einforcement L earning with D ouble Q -learning . arXiv e-prints , arXiv:arXiv: 1509.06461

    2015 arXiv

  50. [58]

    arXiv e-prints , arXiv:arXiv: 1802.02674

    Verma , Siddhartha , Novati , Guido & Koumoutsakos , Petros 2018 Efficient collective swimming by harnessing vortices through deep reinforcement learning . arXiv e-prints , arXiv:arXiv: 1802.02674

    2018 arXiv

  51. [59]

    Vinyals, Oriol , Babuschkin, Igor , Chung, Junyoung , Mathieu, Michael , Jaderberg, Max , Czarnecki, Wojciech M. , Dudzik, Andrew , Huang, Aja , Georgiev, Petko , Powell, Richard , Ewalds, Timo , Horgan, Dan , Kroiss, Manuel , Danihelka, Ivo , Agapiou, John , Oh, Junhyuk , Dal...

    2019

  52. [60]

    1992 Simple statistical gradient-following algorithms for connectionist reinforcement learning

    Williams , Ronald J. 1992 Simple statistical gradient-following algorithms for connectionist reinforcement learning . Machine Learning 8 (3), 229--256

    1992

  53. [61]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sen...

  54. [62]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.