Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

Deep Reinforcement Learning for Active Flow Control around a Three-Dimensional Flow-Separated Wing at Re = 1,000

T0 review · 3 major / 4 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Deep reinforcement learning discovers a jet control for a three-dimensional separated wing that cuts mean drag by 21% and suppresses lift oscillations.

desk verdict First DRL AFC on a 3D wing, but an impossible ΔCl,rms = −124% and a single unseeded run make the central numbers unreliable. read the letter →

arxiv 2509.10195 v1 pith:YUPS4USO submitted 2025-09-12 cs.CE

R. Montalà , B. Font , P. Suárez , J. Rabault , O. Lehmkuhl , R. Vinuesa , I. Rodriguez This is my paper · ORCID
classification cs.CE
keywords deepreinforcementlearningactiveflowcontrolNACA0012wingseparationliftoscillationsdragreductionproximalpolicyoptimizationmulti-agent
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper aims to show that deep reinforcement learning (DRL) can discover effective active flow control for a three-dimensional wing in a separated-flow regime. A PPO-based agent controls a pair of jets on a NACA0012 wing at Reynolds number 1,000 and 20 degrees angle of attack, where vortex shedding and three-dimensional wake structures are fully developed. By the end of training, the learned strategy reduces mean drag by 21% and, as reported, cuts the root-mean-square lift oscillations by 124% relative to the uncontrolled baseline. The agent learns a periodic forcing that locks to the vortex-shedding frequency and delays shear-layer breakdown, leaving a more organized, less three-dimensional wake. This matters because it extends DRL flow control from 2D and canonical cases to a 3D separated wing, a step toward realistic aerodynamic applications.

What carries the argument

The key machinery is the closed-loop DRL control loop: a Proximal Policy Optimization (PPO) agent receives 270 witness-point pressure values (three spanwise slices per pseudo-environment) and outputs a jet velocity, applied to a front/rear jet pair with opposite mass flow to ensure instantaneous mass conservation. The reward function rewards drag reduction and penalizes lift oscillations. Training is accelerated by ten parallel CFD simulations, each split into three pseudo-environments (30 trajectories per step), and by a GPU-accelerated spectral-element solver (SOD2D) communicating with the TF-Agents DRL library through a Redis/SmartSim in-memory layer. The learned policy is then run determ

What would settle it

Re-run the same DRL training with five different random seeds and measure the distribution of drag reduction. If the 21% reduction is not consistently reproduced (e.g., some seeds give no reduction or a drag increase), the central claim collapses. Alternatively, compute ΔCl,rms directly from the raw lift signals as (Cl,rms,ctrl − Cl,rms,base)/Cl,rms,base: if the controlled RMS is not smaller than the baseline RMS, the '−124%' claim fails on its own terms.

Watch

Extended reading notes

Core claim

The paper's central claim is that a DRL agent, using only local surface pressure information from witness points, learns a jet actuation strategy that nearly eliminates lift fluctuations and reduces drag by 21% on a three-dimensional NACA0012 wing at Re=1,000 and AoA=20°, where vortex shedding and three-dimensional wake structures are fully developed. The learned control maintains a negative (suction) mean on the leading-edge jet and a positive (blowing) mean on the rear jet, with an oscillation frequency matching the vortex-shedding Strouhal number St=0.64. This periodic in-phase forcing extends the leading-edge shear layer, delays vortex shedding, and reduces the spanwise disorder of the w

Load-bearing premise

The central claim rests on the assumption that a single unseeded PPO training run is representative, so the observed 21% drag reduction and lift-oscillation suppression are not just run-to-run variance.

Editorial extensions

If this is right

  • If the result is correct, DRL can discover effective active-flow-control strategies for three-dimensional separated wings, not just 2D airfoils or cylinders.
  • The learned control, which locks to the shedding frequency, suggests that DRL can identify the dominant flow instability and exploit it with a periodic forcing that includes additional harmonic components.
  • The framework's scalability—ten parallel simulations and thirty simultaneous trajectories—demonstrates a practical path toward higher Reynolds numbers and more complex geometries.
  • The suppression of lift oscillations and drag reduction could translate into improved aerodynamic efficiency and reduced structural vibration in real applications.
  • The method's success at Re=1,000 invites testing at higher Reynolds numbers, where separation control is more challenging and industrially relevant.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • As stated, the −124% reduction in Cl,rms exceeds the mathematically possible range for a reduction relative to baseline; the authors likely use a different normalization, and a direct comparison of raw RMS values would resolve the actual oscillation suppression.
  • The reported 21% drag reduction comes from a single unseeded training run; repeating training with multiple random seeds would establish whether the learned policy is representative or merely a favorable initialization.
  • Because the actions lock to the vortex-shedding frequency, a natural test is to compare the learned nonlinear policy against a simple sinusoidal forcing at St=0.64; this would quantify the added value of the policy's extra harmonic components.
  • The spanwise periodic pattern of actuation may depend on the periodic boundary conditions and spanwise domain length; testing in a wider domain would probe the robustness of the control in a more realistic configuration.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents a deep reinforcement learning (DRL) framework for active flow control (AFC) around a three-dimensional NACA0012 wing at Re=1,000 and AoA=20°. The environment is a GPU-accelerated spectral-element CFD solver (SOD2D) coupled to the TF-Agents DRL library through Redis/SmartSim, with a multi-environment, multi-agent setup. The agent actuates two spanwise-distributed jet pairs (three pseudo-environments) with a reward combining drag reduction and lift-oscillation penalty. The authors report that the learned policy reduces mean drag by 21% and lift-coefficient RMS by 124%, shifts the vortex-shedding Strouhal number from 0.56 to 0.64, and produces an actuation signal locked to the shedding frequency. They interpret these results as evidence that DRL can discover effective AFC strategies on three-dimensional separated wings.

Significance. If the reported results are correct, this would be a useful demonstration of DRL-based active flow control on a three-dimensional wing with separation, extending previous two-dimensional studies. The computational framework—GPU-accelerated solver, multi-environment MARL training, and reward decomposition—is a constructive engineering contribution, and the validation of the baseline against published airfoil data is a positive feature. However, the central quantitative claims are not presently credible. The reported ΔCl,rms = −124% is mathematically impossible for a nonnegative RMS quantity, and the drag reduction rests on a single unseeded PPO run with no statistical characterization. The scientific conclusion therefore cannot be accepted as stated, despite the soundness of the general methodology.

major comments (3)
  1. [§3, Fig. 5 and §4] The reported lift-oscillation reduction ΔCl,rms = −124% is physically impossible. Since Cl,rms is nonnegative by definition, the relative reduction (Cl,rms,ctrl − Cl,rms,base)/Cl,rms,base cannot be less than −100%. This value is repeated in the Conclusions and must indicate a sign error, a denominator error, or a comparison against an inconsistent baseline window. The manuscript gives no raw time series or computation formula to resolve the discrepancy. This is a load-bearing error: the first central quantitative claim is invalid as stated.
  2. [§3, Figs. 4–5; §2.1] The reported 21% mean-drag reduction is based on a single trained policy obtained from one unseeded PPO run. DRL training is stochastic, and the paper does not report multiple seeds, error bars, or run-to-run variability. Without this information, the 21% reduction cannot be distinguished from the variance of a single training trajectory, which is a known failure mode in RL flow control. The absence of a sensitivity study for the hyperparameters α, β, and the actuator velocity bounds further weakens the claim that the observed performance is robust rather than a lucky initialization.
  3. [§3, Fig. 5 and text near 'root-mean-square'] The paper states that after the transient period 'the root-mean-square of the signal has been noticeably reduced' but the reported −124% implies a reduction beyond complete elimination of oscillations. Even setting aside the arithmetic, the manuscript does not specify how Cl,rms is computed over the controlled and baseline windows (sampling period, number of vortex-shedding cycles, detrending, or window alignment). A proper window-matched RMS calculation with confidence intervals is needed to support any quantitative comparison of oscillation amplitudes.
minor comments (4)
  1. [Abstract/Introduction] Typo: 'limitted' should be 'limited' in the Introduction. Also, the sentence 'in the present work we propose to advance AFC-DRL applicability' reads awkwardly; consider revising.
  2. [References] Reference [17] is attributed to 'F. Schulman' but the correct first author is John Schulman for 'Proximal Policy Optimization Algorithms'.
  3. [Figures] Figures 4 and 5 show training curves and temporal signals, but no error bars or confidence intervals are provided. Adding shading or error bands would help assess convergence and variability. Also, the caption of Fig. 3 lists panels (a)–(d), but panels (a)–(c) are cited in the text; please ensure the panel definitions match.
  4. [§2.2] The term 'flow-throughs' is used to describe the transient period, but a precise definition (e.g., convective time units c/U∞) would improve clarity. The paper uses 'time units' without defining the nondimensionalization.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported control improvements are empirical outcomes of a trained policy, not tautological consequences of the reward definition.

full rationale

The paper's derivation chain is: define the reward function (Eq. 1) to reward drag reduction and penalize lift oscillations; train a PPO policy; then measure Cd, Cl,rms, and the shedding Strouhal number in the CFD environment. This is an optimization result, not a first-principles derivation. The reward is the objective, but it does not by construction guarantee that the agent achieves ΔCd = −21% or ΔCl,rms = −124%; the policy could fail to learn, and the reported numbers are the outcome of the trained policy evaluated in the simulator. The emergent frequency-locking at St = 0.64 and the observed wake-structure changes (Fig. 7) are not encoded in the reward, providing independent empirical content. Baseline comparisons against Gupta et al. [22] and Kouser et al. [23] are external. The self-citations ([7], [8], [15], [16]) concern the MARL/software infrastructure and are not load-bearing for the physical claim. The physically impossible ΔCl,rms = −124% (Section 3) is a serious internal consistency/correctness problem, but it is not a circularity: it does not reduce a prediction to an input by construction. The lack of multiple seeding is a robustness limitation, not a circularity. Therefore the circularity score is 0.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new physical entities. Its central result depends on hand-chosen reward weights and actuator bounds, and on the assumed fidelity of the CFD solver and the single training run.

free parameters (3)
  • α (lift oscillation penalty weight) = 0.3
    Chosen by hand in Eq. 1 to balance drag reduction against lift-oscillation suppression; results depend on this value.
  • β (local vs global reward weight) = 0.8
    Chosen by hand in Eq. 2 to weight local pseudo-environment reward over the global average.
  • Actuator velocity bounds = ±1.125 U∞
    Imposed limits on jet velocity; constrain the policy's action space.
assumptions (3)
  • domain assumption SOD2D accurately solves the incompressible Navier-Stokes/LES equations for this flow at Re=1000.
    Section 2.1: the solver is used without a subgrid model description; baseline validation against literature (Fig. 3) supports but does not guarantee controlled-case accuracy.
  • domain assumption Spanwise-periodic boundary conditions with three pseudo-environments capture the relevant 3D physics.
    Section 2.2: the domain length Lz=3c with three jets; the flow may have spanwise modes longer than 3c not represented.
  • domain assumption PPO converges to a stable policy within 67 episodes.
    Section 3: a single training run is stopped when reward plateaus; no evidence that a rerun or different seed converges similarly.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Deep Reinforcement Learning for Active Flow Control around a Three-Dimensional Flow-Separated Wing at Re = 1,000." pith.science (2026). https://pith.science/paper/YUPS4USO

@misc{pith2026250910195,
  author       = {Pith},
  title        = {Pith review of: Deep Reinforcement Learning for Active Flow Control around a Three-Dimensional Flow-Separated Wing at Re = 1,000},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YUPS4USO}},
  note         = {Machine review of arXiv:2509.10195}
}
read the original abstract

This study explores the use of deep reinforcement learning (DRL) for active flow control (AFC) to reduce flow separation on wings at high angles of attack. Concretely, here the DRL agent controls the flow over the three-dimensional NACA0012 wing section at the Reynolds number Re = 1,000 and angle of attack AoA = 20 degrees, autonomously identifying optimal control actions through real-time flow data and a reward function focused on improving aerodynamic performance. The framework integrates the GPU-accelerated computational fluid dynamics (CFD) solver SOD2D with the TF-Agents DRL library via a Redis in-memory database, enabling rapid training. This work builds on previous DRL flow-control studies, demonstrating DRL potential to address complex aerodynamic challenges and push the boundaries of traditional AFC methods.

Figures

Figures reproduced from arXiv: 2509.10195 by the authors.

Figure 1
Figure 1. CFD-DRL set-up. optimal actuation strategies. The Proximal Policy Optimization (PPO) algorithm [17] is employed to refine the agent’s policy over multiple episodes. The environment is modelled using the SOD2D solver [18], a GPU-enabled spectral element method (SEM) code developed at the Barcelona Supercomputing Center (BSC). This solver is designed for high-fidelity simulations and efficiently solves the filtered in… view at source ↗
Figure 2
Figure 2. Configuration of the domain. The left image illustrates the general setup, while [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. (a) Lift coefficient, (b) drag coefficient and (c) Strouhal number of the vortex [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Evolution of the (a) root-mean-square lift coefficient, (b) drag coefficient, and [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: (a) Lift coefficient and (b) drag coefficient temporal signals. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: (a) Instantaneous jet velocities applied in each pseudo-environment, and (b) [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Q-criterion iso-contours colored by velocity magnitude, along with the instan [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Plug-and-Play Benchmarking of Reinforcement Learning Algorithms for Large-Scale Flow Control

    cs.LG 2026-01 conditional novelty 6.0 of 10

    FluidGym is a PyTorch-only, fully differentiable benchmark with 13 flow-control environments, MARL support, and public baselines.

Reference graph

Works this paper leans on

23 extracted references · 8 canonical work pages · cited by 1 Pith paper

  1. [1]

    Cerutti, C

    J.J. Cerutti, C. Sardu, G. Cafiero and G. Iuso (2020). Active Flow Control on a Square-Back Road Vehicle. Fluids 5, 55. https://doi.org/10.3390/fluids5020055

    doi:10.3390/fluids5020055 2020

  2. [2]

    Minelli, S

    G. Minelli, S. Krajnovi´ c, B. Basara and B.R. Noack (2016). Numerical Investigation of Active Flow Control Around a Generic Truck A-Pillar. Flow, Turbulence and Combustion 97, 1235–1254. https://doi.org/10.1007/s10494-016-9760-3

    work page doi:10.1007/s10494-016-9760-3 2016

  3. [3]

    Rodriguez, O

    I. Rodriguez, O. Lehmkuhl and R. Borrell (2020). Effects of the Actuation on the Boundary Layer of an Airfoil at Reynolds Number Re = 60000. Flow, Turbulence and Combustion 105, 607–626. https://doi.org/10.1007/s10494-020-00160-y

    work page doi:10.1007/s10494-020-00160-y 2020

  4. [4]

    Garnier, J

    P. Garnier, J. Viquerat, J. Rabault, A. Larcher, A. Kuhnle and E. Hachem (2021). A review on deep reinforcement learning for fluid mechanics. Computers & Fluids 225, 104973. https://doi.org/10.1016/j.compfluid.2021.104973

    arXiv 2021

  5. [5]

    Rabault, M

    J. Rabault, M. Kuchta, A. Jensen, U. R´ eglade and N. Cerardi (2019). Artifi- cial neural networks trained through deep reinforcement learning discover con- trol strategies for active flow control. Journal of Fluid Mechanics 865:281–302. https://doi.org/10.1017/jfm.2019.62

    doi:10.1017/jfm.2019.62 2019

  6. [6]

    Rabault and A

    J. Rabault and A. Kuhnle (2019). Accelerating deep reinforcement learning strate- gies of flow control through a multi-environment approach. Physics of Fluids 31 (9): 094105. https://doi.org/10.1063/1.5116415 10

    doi:10.1063/1.5116415 2019

  7. [7]

    Flow control of three-dimensional cylinders transitioning to turbulence via multi-agent reinforcement learning

    P. Su´ arez, F.´Alcantara-´Avila, J. Rabault, A. Mir´ o, B. Font, O. Lehmkuhl and R. Vin- uesa (2025). Flow control of three-dimensional cylinders transitioning to turbulence via multi-agent reinforcement learning. https://doi.org/10.48550/arXiv.2405.17210

    work page Pith review arXiv doi:10.48550/arxiv.2405.17210 2025

  8. [8]

    Su´ arez, F

    P. Su´ arez, F. ´Alcantara-´Avila, A. Mir´ o, J. Rabault, B. Font, O. Lehmkuhl and R. Vinuesa (2025). Active flow control for drag reduction through multi-agent rein- forcement learning on a turbulent cylinder atRe D = 3900. Flow, Turbulence and Combustion. https://doi.org/10.1007/s10494-025-00642-x

    doi:10.1007/s10494-025-00642-x 2025

Show all 23 references
  1. [9]

    Guastoni, J

    L. Guastoni, J. Rabault, P. Schlatter, H. Azizpour and R. Vinuesa (2023). Deep reinforcement learning for turbulent drag reduction in channel flows. The European Physical Journal E 46, 27. https://doi.org/10.1140/epje/s10189-023-00285-8

    2023 doi

  2. [10]

    Sonoda, Z

    T. Sonoda, Z. Liu, T. Itoh and Y. Hasegawa (2023). Reinforcement learning of control strategies for reducing skin friction drag in a fully developed turbulent channel flow. Journal of Fluid Mechanics 960:A30. https://doi.org/doi:10.1017/jfm.2023.147

    2023 doi

  3. [11]

    Cavallazzi, L

    G.M. Cavallazzi, L. Guastoni, R. Vinuesa and A. Pinelli (2024). Deep Reinforcement Learning for the Management of the Wall Regeneration Cycle in Wall-Bounded Tur- bulent Flows. Flow, Turbulence and Combustion. https://doi.org/10.1007/s10494- 024-00609-4

    2024 doi

  4. [12]

    Vasanth, J

    J. Vasanth, J. Rabault, F. Alc´ antara- ´Avila, M. Mortensen and R. Vin- uesa (2024). Multi-agent Reinforcement Learning for the Control of Three- Dimensional Rayleigh–B´ enard Convection. Flow, Turbulence and Combustion. https://doi.org/10.1007/s10494-024-00619-2

    2024 doi

  5. [13]

    Wang, Y.-F

    Y.-Z. Wang, Y.-F. Mei, N. Aubry, Z. Chen, P. Wu and W.-T. Wu (2022). Deep rein- forcement learning based synthetic jet control on disturbed flow over airfoil. Physics of Fluids 34 (3): 033606. https://doi.org/10.1063/5.0080922

    2022 doi

  6. [14]

    Garcia, A

    X. Garcia, A. Mir´ o, P. Su´ arez, F.´Alcantara-´Avila, J. Rabault, B. Font, O. Lehmkuhl and R. Vinuesa (2025). Deep-reinforcement-learning-based separation control in a two-dimensional airfoil. https://doi.org/10.48550/arXiv.2502.16993

    2025 doi arXiv

  7. [15]

    B. Font, F. Alc´ antara-´Avila, J. Rabault, R. Vinuesa and O. Lehmkuhl (2025). Deep reinforcement learning for active flow control in a turbulent separation bubble. Nature Communications 16, 1422. https://doi.org/10.1038/s41467-025-56408-6

    2025 doi

  8. [16]

    Montal` a, B

    R. Montal` a, B. Font, O. Lehmkuhl, R. Vinuesa and I. Rodr ´ ıguez (2024). Towards Ac- tive Flow Control Strategies Through Deep Reinforcement Learning, in: 9th ECCO- MAS, 3rd-7th June, Lisbon (Portugal). https://doi.org/10.23967/eccomas.2024.115

    2024 doi

  9. [17]

    Schulman, P

    F. Schulman, P. Wolski, A. Dhariwal, A. Radford and O. Klimov (2017). Proximal policy optimization algorithms. https://doi.org/10.48550/arXiv.1707.06347 11

    2017 doi arXiv

  10. [18]

    Gasparino, F

    L. Gasparino, F. Spiga and O. Lehmkuhl (2024). SOD2D: A GPU-enabled spectral fi- nite elements method for compressible scale-resolving simulations. Computer Physics Communications 297, 109067. https://doi.org/10.1016/j.cpc.2023.109067

    2024

  11. [19]

    Guadarrama, A

    S. Guadarrama, A. Korattikara, O. Ramirez, P. Castro, E. Holly, S. Fishman, K. Wang, E. Gonina, N. Wu, E. Kokiopoulou, L. Sbaiz, J- Smith, G. Bart, J. Berent, C. Harris, V. Vanhoucke and E. Brevdo (2018). TF-Agents: A library for Reinforcement Learning in TensorFlow. https://g...

    2018

  12. [20]

    Partee, M

    S. Partee, M. Ellis, A. Rigazzi, A.E. Shao, S. Bachman, G. Marques and B. Rob- bins (2022). Using machine learning at scale in HPC simulations with Smart-Sim: An application to ocean climate modeling. Journal of Computer Science 62:101707. https://doi.org/10.5281/zenodo.4682270

    2022 doi

  13. [21]

    Belus, J

    V. Belus, J. Rabault, J. Viquerat, Z. Che, E. Hachem and U. Reglade (2019). Ex- ploiting locality and translational invariance to design effective deep reinforcement learning control of the 1-dimensional unstable falling liquid film. AIP Advances 9 (12): 125014. https://doi.or...

    2019 doi

  14. [22]

    Gupta, J

    S. Gupta, J. Zhao, A. Sharma, A. Agrawal, K. Hourigan and M.C (2023). Thompson. Two- and three-dimensional wake transitions of a NACA0012 airfoil. Journal of Fluid Mechanics 954:A26. https://doi.org/10.1017/jfm.2022.958

    2023 doi

  15. [23]

    Kouser, Y

    T. Kouser, Y. Xiong, D. Yang and S. Peng (2021). Direct Numerical Sim- ulations on the three-dimensional wake transition of flows over NACA0012 airfoil at Re = 1000. International Journal of Micro Air Vehicles 13. https://doi.org/10.1177/17568293211055656 12

    2021 doi

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.