Pith. sign in

REVIEW 4 major objections 5 minor 36 references

Reinforcement-learning-assisted control of four-roll mills: geometric symmetry and inertial effect

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A deep reinforcement-learning controller can steer a displaced droplet back to the centre of a four-roll mill at Reynolds numbers up to 3, and a policy trained in one of the mill's eight sectors controls all eight via geometric symmetry.

desk verdict Finite-Re DRL control of a four-roll mill is credible and useful, but missing seeds and artifacts plus a slightly overbroad robustness claim make the paper conditional, not a knockout. read the letter →

arxiv 2504.20336 v2 pith:FPEJUY6M submitted 2025-04-29 physics.flu-dyn

Xuan Dai , Da Xu , Mengqi Zhang , Yantao Yang This is my paper · ORCID
classification physics.flu-dyn
keywords reinforcementlearningfour-rollmillflowcontrolgeometricsymmetryinertialeffectsdroplettrajectoryextensionalproximalpolicyoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that a deep reinforcement-learning agent can steer a small rigid droplet back to the centre of a four-roll mill even when the surrounding flow has mild inertia, at Reynolds numbers up to 3, using only the droplet's position, velocity, and acceleration as inputs. It also claims that the mill's eight-fold geometric symmetry makes the task eight times cheaper: a control policy trained in one of the eight sub-quadrants can be reflected and rotated to control droplets in the other seven with no retraining. A sympathetic reader would care because this is the first demonstration of learned feedback control for this classic extensional-flow device in a nonlinear regime, and because the symmetry trick is a general prescription, not a special hack. The paper further argues that inertia is the limiting factor: it delays the flow's response to roller-speed changes, and when the delay is too large, at Reynolds number 5 in their tests, the action-reward link breaks and control fails.

What carries the argument

The engine of the argument is the equivariance condition $r(s,a)=r(L_g[s], K_g^s[a])$, where $L_g$ maps a droplet's state, consisting of position, velocity, and acceleration, in one sub-quadrant to the equivalent state in another, and $K_g^s$ maps the four roller-speed multipliers accordingly. This identity is what lets a policy learned in one sector be applied to the other seven by composing the network with $L_g$ and $K_g^s$. The second piece is the reward function $r(t)=\exp[-p(1-\cos\beta(t))]+\exp[-q\,h(t)]+r'$, where $\beta$ is the angle between the droplet's latest displacement and the inward radial direction and $h(t)$ is its distance from the centre; together with a terminal bonus $r'$ for reaching $h\le h_e$, this reward gives PPO a well-posed credit-assignment problem. The action interval $\Delta t_a$ is chosen so that it is at least as large as the measured flow-response time $\Delta t_d$, which is what makes the delayed inertial response observable to the agent.

What would settle it

Measure $\Delta t_d$ at $Re=5$, set $\Delta t_a$ well above it while keeping the same six-dimensional state and reward, and train the same PPO agent; if it drives the droplet to $h\le h_e$, the paper's claim that inertial delay fundamentally breaks the action-reward link is incorrect. In parallel, apply the $L_g$ and $K_g^s$-transformed policy from sub-quadrant iii to all other sectors at $Re=3$; any sector whose final distance exceeds $h_e$ disproves the symmetry-transfer claim.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that proximal policy optimization (PPO), trained against direct numerical simulations of the Navier-Stokes equations, can reliably guide a displaced droplet into a target radius $h_e=0.005$ from initial distances as large as $h_0=0.25\sqrt{2}$, for $Re=10^{-9}$, $0.4$, $2$, and $3$. The state is the six-dimensional vector $[x,y,u,v,k_x,k_y]$; a position-only state fails in the finite-$Re$ cases, while dropping acceleration still works at $Re=2$. The control acts on the two rollers adjacent to the droplet, or three rollers at $Re=3$, and the same converged policy, transformed by the symmetry operators $L_g$ and $K_g^s$, successfully controls droplets in all eight sub-quadrants without retraining. The authors also establish that inertial delay of the flow response is what ultimately limits control, and they set the action-update interval $\Delta t_a$ from a measured response time so that each action's effect is visible before the next action is taken.

Load-bearing premise

The load-bearing premise is that the six numbers describing the droplet's position, velocity, and acceleration at the chosen action interval are a complete description of the control problem, so if the inertial delay hides any needed information, the claimed robust control is not guaranteed outside the tested regimes.

Editorial extensions

If this is right

  • Training cost drops by a factor of eight: one converged policy in sub-quadrant iii transfers to all eight sub-quadrants, and the authors demonstrate that the transformed trajectories reach the same target radius.
  • The controllable range roughly quintuples relative to the earlier Stokes-flow controller: initial distances up to $h_0=0.25\sqrt{2}$ are handled, compared with starting positions within about $0.05\sqrt{2}$ of the centre.
  • At finite Reynolds number the state must include velocity, and at $Re=3$ a third roller is needed; both changes are direct consequences of the inertial response delay.
  • There is a hard ceiling in this framework: $Re=5$ cannot be controlled even with the enriched state, because the flow response is so delayed that actions appear to have no effect.
  • Setting the action interval $\Delta t_a$ from a measured response time $\Delta t_d$ is a practical design rule for any DRL flow controller in a regime with inertia.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the symmetry transfer relies only on the reward-invariance identity, not on PPO or on neural networks, the same eight-sector lift should work for any control method, including PID, model-predictive control, or other reinforcement-learning algorithms, on a four-roll mill.
  • The $Re=5$ failure suggests the next lever is not more rollers but a history-aware state, such as a short window of past positions and velocities or a learned delayed-state embedding, since the paper's own ad hoc tests show the droplet keeps moving along the old trajectory after an action change.
  • The rigid-droplet assumption is doing quiet work: for deformable drops the shape breaks the symmetry, so the eight-sector transfer and the learned policy are likely to degrade exactly where breakup and interfacial physics start to matter.
  • A cheap experimental check of the symmetry claim would be to run a single trained policy in one sector, flip the roller-speed protocol according to $K_g^s$, and verify that the droplet trajectory mirrors in the other sector; any mismatch would identify where the DNS or rigid-drop assumption breaks.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This paper extends reinforcement-learning control of a rigid passive droplet in a two-dimensional four-roll mill from the Stokes regime to finite-Reynolds-number flows, using direct numerical simulation with Nek5000. The trained PPO agents use a six-dimensional state of droplet position, velocity, and acceleration, and act by modulating the rotation speeds of two or, at Re=3, three adjacent rollers. A reward combining directional memory, radial distance, and a terminal bonus is used, and the geometric symmetry of the mill is exploited to transfer policies trained in one of eight sub-quadrants to the whole domain. The authors report successful controlled trajectories for five initial conditions at Re=10^-9, 0.4, 2, and 3, and provide appendices on the action-update interval, noisy environments, global-policy generalization, reward hyperparameters, and state definitions.

Significance. If the reported results hold, the paper makes a useful contribution: it demonstrates DRL control of an unstable extensional flow outside the Stokes limit, quantifies the role of inertial response delay, and gives an explicit, exact symmetry recipe for policy transfer across sub-quadrants. The DNS validation is quantitative and convincing (extension rate 0.7065 versus Higdon's 0.7064, vorticity reproduced exactly), and the symmetry transformations in Eqs. (3.1)-(3.4) are clearly specified and exact. The appendices are candid about what does not work (Re=5, position-only states, acceleration ablation), which is a methodological strength. However, the central evidence base is thin: policies are trained once per case, the success criterion coincides with the shaped reward, and the sufficiency of the local state is asserted rather than demonstrated. The significance of the robustness claim is therefore conditional on additional statistical and state-sufficiency evidence.

major comments (4)
  1. [§3.1, Figs. 2-3 and Table 1] The claim of robust, successful control is supported only by single PPO runs per case. The convergence criterion is stated as 'more than 30 to 50 consecutive epochs of successful control', and the reported trajectories in Figures 3, 6, 8, and 9 are single realizations; the 'additional 50 epochs' are a continuation of the same training run rather than repeated independent seeds. Because PPO is stochastic, the reader cannot determine whether the displayed trajectories are representative or outliers. Please report repeated-seed statistics (at least several seeds per case and per symmetry-transferred octant), including success rates and final-distance distributions, for both training and evaluation.
  2. [Appendix A and Table 1, case 5.4] The paper's own heuristic is that the action update interval Delta_t_a should be 'of the same order as, or larger than' the flow response time Delta_t_d. For Re=3, Figure 10(d) reports Delta_t_d up to 0.0893 at the farthest probe (h0=0.25*sqrt(2), alpha0=45 degrees), while Table 1 sets Delta_t_a=0.075 for case 5.4. The interval is therefore slightly smaller than the measured response time at the very position used in that case, leaving the Re=3 result marginal by the authors' own criterion. Since Delta_t_a is the mechanism by which the six-dimensional local state is intended to remain a sufficient Markov description, please either revise the heuristic, choose Delta_t_a so that it safely exceeds the response time, or provide a sensitivity study showing that the Re=3 policy is robust to changes in Delta_t_a near this boundary.
  3. [§2.3, Appendix E, §3.3.2] The sufficiency of the local state s_t=[x,y,u,v,kx,ky] for the Navier-Stokes control problem is not established beyond the specific trained cases. The paper itself shows in Appendix E that acceleration is dispensable at Re=2 and reports that even the full local state fails at Re=5, indicating that the problem sits at the edge of the state representation's validity. Because the headline robustness claim and the symmetry-transfer claim both inherit this Markov assumption, please provide a more direct test of state sufficiency at Re=3, for example by comparing policies trained with different Delta_t_a values, or with a history-augmented state, and by reporting how the success rate degrades as the action interval approaches the response time.
  4. [§2.2, Eq. (2.2), §3.1] The reported success criterion (minimum distance below h_e) is the same quantity that is directly optimized by the reward through r2(t)=exp(-q h(t)) and the terminal bonus r'=c for h(t)<=h_e. This is not circular in a strict logical sense, but it means the central claim of robust performance is evaluated on the shaped objective rather than on an independent control metric. Please supplement the results with out-of-sample evaluation: held-out initial positions not used in training, repeated stochastic evaluation episodes, and separate reporting of final-distance statistics and success rates rather than training-history minima.
minor comments (5)
  1. [§2.1] The rigid-droplet assumption is inherited from Vona & Lauga, but at Re=3 droplet deformation and back-coupling may no longer be negligible; please state explicitly why the assumption is valid for the parameter range studied or add it to the limitations.
  2. [Figure 10 caption] The caption states that probes are placed at h0=[0.25*sqrt(2), 0.05*sqrt(2), 0.25*sqrt(2)] with alpha0=45 degrees, which duplicates 0.25*sqrt(2); the text lists 0.05, 0.15, and 0.25, so the caption should be corrected.
  3. [Appendix B] The added forcing in Eq. (B 1) is described as thermal noise, but no fluctuation-dissipation relation is included in the Navier-Stokes solver; 'stochastic perturbation' or 'synthetic noise' would be more accurate terminology.
  4. [Appendix C] The final paragraph claims that a trained policy can be transferred to similar settings and geometries, supported only by 'results not shown'; this unsupported statement should either be removed or accompanied by the corresponding data.
  5. [Abstract and §3.1] The abstract's phrase 'robust performance across various starting positions' is stronger than what is shown, since each policy is trained separately for its own initial condition and only a few positions are tested; please qualify the claim to the tested cases.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity; the symmetry map is an exact group action and the control results are empirical DNS tests, not fitted predictions.

full rationale

The paper's derivation chain is self-contained and non-circular. The central claims are (i) PPO with state s_t=[x,y,u,v,kx,ky] and reward Eq. (2.2) produces policies that drive droplets to the target in DNS, and (ii) the eight-fold geometric symmetry lets a policy trained in sub-quadrant iii be mapped to the other seven sub-quadrants. Neither claim reduces to an input by construction. The state, reward, and action spaces are defined independently of the reported success; success is measured by the simulated trajectory's final distance h(t)<=h_e, which is an external DNS outcome rather than a quantity fitted from the trained policy. The reward does directly penalize distance via r2(t)=exp[-q h(t)] and r', but that is an optimization objective, not a prediction: the paper does not claim to derive the control capability from the reward alone, and the converged policies are tested in the flow solver. The symmetry argument is an exact group action: the L_g matrices in Eqs. (3.1)-(3.2) are coordinate reflections of the domain, and the K_g matrices in Eqs. (3.3)-(3.4) permute roller actions accordingly; these are mathematical identities for the symmetric four-roller geometry, not fitted relationships. The validation in Fig. 6 applies the transformed policy to other sub-quadrants without retraining and checks the DNS trajectories, which is an independent test. The only self-citations (Li & Zhang 2022; Xu & Zhang 2023) are routine examples of PPO/DDPG usage and are not load-bearing. The paper also discloses its own limits: Re=5 control fails and Appendix E shows acceleration is dispensable at Re=2; these are honesty caveats, not circular reductions. No equation is fitted and then renamed as a prediction, no uniqueness theorem is imported from the authors' prior work, and no ansatz is smuggled via citation. Score 0.

Assumptions & free parameters 9 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the numerical fidelity of the DNS, the Markov sufficiency of the chosen state, the exactness of the geometric symmetry, and several hand-tuned reward and training hyperparameters. No new physical entity is introduced. The most fragile axiom is the state sufficiency at finite Re, which the authors themselves show breaking at Re = 5.

free parameters (9)
  • Reward exponent p = 2
    Exponent in r1 = exp[-p(1-cos beta)]; chosen in Appendix D by comparing p = 1, 2, 4, 10 for one Re = 0.4 case.
  • Reward exponent q = 30, 10, or 7 depending on h0
    Exponent in r2 = exp[-q h]; tuned per initial distance so the reward range covers the trajectory, as shown in Appendix D.
  • Terminal reward c = 2
    Bonus when h <= h_e; tested c = 2 versus c = 4 in Appendix D and c = 2 selected.
  • Target distance h_e = 0.0025 or 0.005
    User-defined success threshold in Table 1; the success claim depends on this threshold.
  • Action clipping value eta = 1.5, 2, or 3 by case
    Limits sampled action magnitudes; case-dependent in Table 1 with no independent justification.
  • Action update interval Delta t_a = 0.05 or 0.075
    Set from heuristic response-time estimates in Appendix A; the authors note the values are not necessarily optimal.
  • Discount factor gamma = 0.98 or 0.99
    Case-dependent hyperparameter in Table 1; standard RL choice.
  • Maximum steps per epoch N = 30, 50, or 90
    Case-dependent horizon in Table 1; chosen to give enough time for control.
  • Number of controlled rollers = 2 for Re <= 2, 3 for Re = 3
    One roller was too restrictive at finite Re, and Re = 3 required an extra roller, as described in Sections 2.2 and 3.3.2.
assumptions (5)
  • domain assumption The droplet is a rigid fluid particle with no deformation and no back-coupling to the flow.
    Stated in Section 2.1 following Vona and Lauga (2021); removes droplet internal dynamics that could matter at finite Re.
  • domain assumption State [x, y, u, v, kx, ky] with constant actions over Delta t_a makes the control problem Markovian for PPO.
    Assumed in Section 2.3 and Appendix A; the paper's own Re = 5 failure shows this premise has a limited range.
  • domain assumption The square FRM flow and the reward are exactly equivariant under the eight-sector reflection group.
    Used in Eq. (2.3) and Section 3.2; exact for symmetric geometry and no-slip boundaries, but validated only through plotted trajectories.
  • domain assumption The DNS with Nek5000, 1436 order-7 elements, and dt = 10^-3 accurately resolves the finite-Re flow.
    Mesh convergence is claimed in Section 2.1, but quantitative validation is provided only at the Stokes limit against Higdon (1993).
  • ad hoc to paper PPO with the listed networks and hyperparameters converges to a successful policy for the tested cases.
    No proof of convergence; the evidence is empirical training histories in Figure 2 with no repeated seeds.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Reinforcement-learning-assisted control of four-roll mills: geometric symmetry and inertial effect." pith.science (2026). https://pith.science/paper/FPEJUY6M

@misc{pith2026250420336,
  author       = {Pith},
  title        = {Pith review of: Reinforcement-learning-assisted control of four-roll mills: geometric symmetry and inertial effect},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FPEJUY6M}},
  note         = {Machine review of arXiv:2504.20336}
}
abstract

Embedding the intrinsic symmetry of a flow system in training its machine learning algorithms has become a significant trend in the recent surge of their application in fluid mechanics. This paper leverages the geometric symmetry of a four-roll mill (FRM) to enhance its training efficiency. Stabilizing and precisely controlling droplet trajectories in a FRM is challenging due to the unstable nature of the extensional flow with a saddle point. Extending the work of Vona & Lauga, this study applies Deep Reinforcement Learning (DRL) to effectively guide a displaced droplet to the center of the FRM. Through direct numerical simulations, we explore the applicability of DRL in controlling FRM flow with moderate inertial effects, i.e., Reynolds number $\sim\mathcal{O}(1)$, a nonlinear regime previously unexplored. The FRM's geometric symmetry allows control policies trained in one of the eight sub-quadrants to be extended to the entire domain, reducing training costs. Our results indicate that the DRL-based control method can successfully guide a displaced droplet to the target center with robust performance across various starting positions, even from substantially far distances. The work also highlights potential directions for future research, particularly focusing on efficiently addressing the delay effects in flow response caused by inertia. This study presents new advances in controlling droplet trajectories in more nonlinear and complex situations, with potential applications to other nonlinear flows. The geometric symmetry used in this cutting-edge reinforcement learning approach can also be applied to other control methods.

Figures

Figures reproduced from arXiv: 2504.20336 by the authors.

Figure 1
Figure 1. (𝑎) A schematic diagram of four-roll mill showing the control setup for a droplet initially positioned in the sub-quadrant 𝑖𝑖𝑖. (𝑏) The five initial positions of the droplet to be studied in the present work with different ℎ0 and 𝛼0. The blue shade encircles the initial positions considered by Vona & Lauga (2021). The definition of the angle 𝛽 in the reward definition Eq. 2.2 is also illustrated. (𝑐) Validation of o… view at source ↗
Figure 2
Figure 2. Agent training history for droplets at different initial positions. The first row corresponds to the closest radial distance to the origin. The second row shows the cumulative rewards of each epoch. inertial effects and geometric symmetry, is effectively captured within our current numerical framework. This will be demonstrated in the result section. In the end, we explain the other theme of the work, that is how to… view at source ↗
Figure 3
Figure 3. Converged policies in figure 2 are run for 50 episodes at 𝑅𝑒 = 0.4, All droplets are successfully driven to the target radial distance. Panels (a), (c) demonstrate representative trajectories for all the considered cases. Panels (b), (d) display the corresponding actions. Note that only two rollers (1) and (2) are acted on, and an action is followed by a waiting time Δ𝑡𝑎 = 50Δ𝑡 until the next one, during which it is… view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Instantaneous quiver plots for the velocity field in case 5.2 at 𝑅𝑒 = 0.4. (a) Without control. The droplet tends to be swept away. The flow is inherently symmetric. (b) The last time step of an epoch for a converged policy. The white cross represents the starting posi…
Figure 5
Figure 5. Figure 5: The domain of FRM is evenly divided into 8 sub-quadrants. By the geometric symmetry, the control policy trained in one of the sub-quadrants can be applied to the entire domain. For example, panel (𝑎) shows that the dynamics of the blue droplet in sub-quadrant 𝑖𝑖𝑖 and t…
Figure 6
Figure 6. Figure 6: Trajectories by applying the policies trained in section 3.1 based on geometric symmetry in FRM. Note that the polices trained in the 𝑖𝑖𝑖 sub-quadrant (see the solid lines) are applied directly to other sub-quadrants (the dashed lines) without new training. (𝑎) ℎ0 = 0.…
Figure 7
Figure 7. Figure 7: Effect of inertia on the droplet trajectory subject to ad hoc action variation. The blue dot marks the initial position of the droplet at [−0.25, 0.25]. Panel (𝑎) illustrates the action history of the roller (labeled as roller 2), where different colours represent the …
Figure 8
Figure 8. Figure 8: Droplet trajectory under control corresponding to the case 5.1 in table 1 with the parameters ℎ0 = 0.25√ 2, 𝛼0 = 45◦ and 𝑅𝑒 = 10−9 , compared to the baseline case 5.2 (𝑅𝑒 = 0.4), which has been explained in detail in section 3.1. (𝑏) The roller actions in the case of ℎ…
Figure 9
Figure 9. Figure 9: (𝑎) Controlled droplet trajectory for case 5.3 in [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: The response test across 𝑅𝑒’s. The dashed line represents the actions exerted by roller (2), denoted as 𝑎1 (𝑡). The red, green and blue lines are the velocity signals monitored at positions of ℎ0 = [0.25√ 2, 0.05√ 2, 0.25√ 2] and 𝛼0 = 45◦ . position [PITH_FULL_IMAGE:…
Figure 11
Figure 11. Figure 11: Robustness test based on adding thermal noise to the flow environment. (a) The accuracy as the number of successful controls out of 100 total runs. (b) The average distance of 100 total runs. greatest 𝑅𝑒 (=3) is negatively influenced most by the thermal noise. This is…
Figure 12
Figure 12. Figure 12: Final distances of droplets to the target position. Droplets are initially located at the cell centers on a 10 × 10 rectangular grid, uniformly distributed over 𝑥 × 𝑦 ∈ [−0.3, −0.1] × [0.1, 0.3]. The applied policy was trained for the specific initial position (−0.25,…
Figure 13
Figure 13. Figure 13: The first reward term 𝑟1 (𝑡) against the angle in degrees between two displacement vector. -0.1 -0.05 0.0 0.05 0.1 x -0.1 -0.05 0.0 0.05 0.1 y p = 1 -0.1 -0.05 0.0 0.05 0.1 x -0.1 -0.05 0.0 0.05 0.1 y p = 2 -0.1 -0.05 0.0 0.05 0.1 x -0.1 -0.05 0.0 0.05 0.1 y p = 4 -0.…
Figure 14
Figure 14. Figure 14: The trajectories obtained by policies trained using only 𝑟1 (𝑡) with different choices of 𝑝. The other terms in the reward function, i.e., 𝑟2 (𝑡), 𝑟′ , are excluded in this test. on figure 13. For 𝑝 ⩾ 2, no obvious differences are found in terms of the moving directio…
Figure 15
Figure 15. Figure 15: The second reward term 𝑟2 (𝑡) against the distance to the origin ℎ(𝑡). -0.1 -0.05 0.0 0.05 0.1 x -0.1 -0.05 0.0 0.05 0.1 y p = 2; q = 30 (𝑎) 0 200 400 600 800 1000 Epoch 0.01 0.1 Final distance he = 0:0025 0 200 400 600 800 1000 Epoch 0 5 10 15 20 25 30 35 Retrun (𝑏) …
Figure 16
Figure 16. Figure 16: (a) The trajectory obtained by policy trained with 𝑟1 (𝑡) + 𝑟2 (𝑡) ′ using [𝑝, 𝑞] = [2, 30] without adding 𝑟 ′ ; (bc) The training history. the origin. The reason may be that once the droplet moves in the radial direction pointing towards the target, a high reward is …
Figure 17
Figure 17. Figure 17: (a) The trajectory obtained by policy trained with 𝑟1 (𝑡) + 𝑟2 (𝑡) + 𝑟 ′ using [𝑝, 𝑞, 𝑐] = [2, 30, 2]; (bc) The training history. -0.1 -0.05 0.0 0.05 0.1 x -0.1 -0.05 0.0 0.05 0.1 y p = 2; q = 30; c = 4 (𝑎) 0 200 400 600 800 1000 0.01 0.1 Final distance he = 0:0025 0 …
Figure 18
Figure 18. Figure 18: (a) The trajectory obtained by policy trained with 𝑟1 (𝑡) + 𝑟2 (𝑡) + 𝑟 ′ using [𝑝, 𝑞, 𝑐] = [2, 30, 4]; (bc) The training history. ℎ0 = 0.05√ 2. To verify that this value is effective for the control, we will do a numerical study for 𝑐 = 2 and 𝑐 = 4 [PITH_FULL_IMAGE:f…
Figure 19
Figure 19. Figure 19: (𝑎) Droplet trajectory under control corresponding to the case 5.3 in table 1 with the parameters ℎ0 = 0.25√ 2, 𝛼0 = 45◦ and 𝑅𝑒 = 2. The dashed lines result from the application of geometric symmetry in FRM. (𝑏) Action history and value history in the test. Additional…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

36 extracted references · 26 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence ...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    , Rabault, J

    Belus, V. , Rabault, J. , Viquerat, U. , Che, Z. , Hachem, E. & Reglade, U. 2019 Exploiting locality and translational invariance to design effective deep reinforcement learning control of the 1-dimensional unstable falling liquid film . AIP Advances 9 (12), 125014

    work page 2019

  4. [4]

    Bentley, B. J. & Leal, L. G. 1986 a\/ A computer-controlled four-roll mill for investigations of particle and drop dynamics in two-dimensional linear shear flows . J. Fluid Mech. 167 , 219--240

    work page 1986

  5. [5]

    Bentley, B. J. & Leal, L. G. 1986 b\/ An experimental investigation of drop deformation and breakup in steady, two-dimensional linear flows . J. Fluid Mech. 167 , 241--283

    work page 1986

  6. [6]

    Brunton, S. L. , Noack, B. R. & Koumoutsakos, P. 2020 Machine learning for fluid mechanics . Annu. Rev. Fluid Mech. 52 , 477--508

    2020

  7. [7]

    Bucci, M. A. , Semeraro, O. , Allauzen, A. , Wisniewski, G. , Cordier, L. & Mathelin, L. 2019 Control of chaotic systems by deep reinforcement learning . Proc. R. Soc. A 475 , 20190351

    work page 2019

  8. [8]

    , Yang, L

    Fan, D. , Yang, L. , Wang, Z. , Triantafyllou, M.S. & Karniadakis, G.E. 2020 Reinforcement learning for bluff body active flow control in experiments and simulations . Proc. Natl. Acad. Sci. U.S.A. 117 (42), 26091--26098

    work page 2020

Show all 36 references
  1. [9]

    & Leal, L

    Feng, J. & Leal, L. G. 1997 Numerical simulations of the flow of dilute polymer solutions in a four-roll mill . J. Non-Newton. Fluid Mech. 72 (2-3), 187--218

    1997

  2. [10]

    Fischer, P. F. , Lottes, J. W. & Kerkemeier, S. G. 2017 Nek5000 Version 17.0

    2017

  3. [11]

    , Rallison, J

    Fuller, G.G . , Rallison, J. M. , Schmidt, R. L. & Leal, L. G. 1980 The measurement of velocity gradients in laminar flow by homodyne light-scattering spectroscopy . J. Fluid Mech. 100 (3), 555--575

    1980

  4. [12]

    Fuller, G. G. & Leal, L. G. 1981 Flow birefringence of concentrated polymer solutions in two-dimensional flows . J. Polym. Sci. 19 (4), 557--587

    1981

  5. [13]

    Higdon, J. J. L. 1993 The kinematics of the four-roll mill . Phys. Fluids A 5 (1), 274--276

    1993

  6. [14]

    Hudson, S. D. , Phelan, F. R., Jr. , Handler, M. D. , Cabral, J. T. , Migler, K. B. & Amis, E. J. 2004 Microfluidic analog of the four-roll mill . Appl. Phys. Lett. 85 (2), 335--337

    2004

  7. [15]

    , Kim, H

    Kim, J. , Kim, H. , Kim, J. & Lee, C. 2022 Deep reinforcement learning for large-eddy simulation modeling in wall-bounded turbulence . Phys. Fluids 34 (10), 105132

    2022

  8. [16]

    Kingma, D. P. & Ba, J. 2014 Adam: A method for stochastic optimization . arXiv: 1412.6980

    2014 arXiv

  9. [17]

    Lee, J. S. , Dylla-Spears, R. , Teclemariam, N. P. & Muller, S. J. 2007 Microfluidic four-roll mill for all flow types . Appl. Phys. Lett. 90 (7)

    2007

  10. [18]

    & Zhang, M

    Li, J. & Zhang, M. 2022 Reinforcement-learning-based control of confined cylinder wakes with stability analyses . J. Fluid Mech. 932 , A44

    2022

  11. [19]

    2010 Stretching polymer chains

    Mackley, M. 2010 Stretching polymer chains . Rheol. Acta 49 (5), 443--458

    2010

  12. [20]

    Otto, S. E. , Zolman, N. , Kutz, J. N. & Brunton, S. L. 2023 A unified framework to enforce, discover, and promote symmetry in machine learning . arXiv:2311.00212v1

    2023 arXiv

  13. [21]

    , Worrall, D

    van der Pol, E. , Worrall, D. E. , van Hoof, H. , Oliehoek, F. A. & Welling, M. 2021 MDP Homomorphic Networks: Group Symmetries in Reinforcement Learning . NeurIPS, arXiv:2006.16908

    2021 arXiv

  14. [22]

    , Kuchta, M

    Rabault, J. , Kuchta, M. , Jensen, A. , R \'e glade, U. & Cerardi, N. 2019 Artificial neural networks trained through deep reinforcement learning discover control strategies for active flow control . J. Fluid Mech. 865 , 281--302

    2019

  15. [23]

    Rallison, J. M. 1984 The deformation of small viscous drops and bubbles in shear flows . Annu. Rev. Fluid Mech. 16 , 45--66

    1984

  16. [24]

    , Rabault, J

    Ren, F. , Rabault, J. & Tang, H. 2021 Applying deep reinforcement learning to active flow control in weakly turbulent conditions . Phys. Fluids 33 (3), 037121

    2021

  17. [25]

    Rumscheidt, F. D. & Mason, S. G. 1961 Particle motions in sheared suspensions XI. Internal circulation in fluid droplets (experimental) . J. Colloid Sci. 16 (3), 210--237

    1961

  18. [26]

    , Wolski, F

    Schulman, J. , Wolski, F. , Dhariwal, P. , Radford, A. & Klimov, O. 2017 Proximal policy optimization algorithms . arXiv:1707.06347

    2017 arXiv

  19. [27]

    Stone, H. A. 1994 Dynamics of drop deformation and breakup in viscous fluids . Annu. Rev. Fluid Mech. 26 , 65--102

    1994

  20. [28]

    , \'A lcantara- \'A vila, F

    Su \'a rez, P. , \'A lcantara- \'A vila, F. , Mir \'o , A. , Rabault, J. , Font, B. , Lehmkuhl, O. & Vinuesa, R. 2024 a\/ Active flow control for drag reduction through multi-agent reinforcement learning on a turbulent cylinder at red = 3900 . arXiv: 2405.17655

    2024 arXiv

  21. [29]

    , \'A lcantara- \'A vila, F

    Su \'a rez, P. , \'A lcantara- \'A vila, F. , Rabault, J. , Mir \'o , A. , Font, B. , Lehmkuhl, O. & Vinuesa, R. 2024 b\/ Flow control of three-dimensional cylinders transitioning to turbulence via multi-agent reinforcement learning . arXiv: 2405.17210

    2024 arXiv

  22. [30]

    Sutton, S. R. & Barto, A. G. 2018 Reinforcement learning: An introduction\/ . The MIT Press

    2018

  23. [31]

    Taylor, G. I. 1934 The formation of emulsions in definable fields of flow . Proc. R. Soc. Lond. A 146 (858), 501--523

    1934

  24. [32]

    , Rabault, J

    Vasanth, J. , Rabault, J. , Alc \'a ntara- \'A vila, F. , Mortensen, M. & Vinuesa, R. 2024 Multi-agent reinforcement learning for the control of three-dimensional Rayleigh-B \'e nard convection . arXiv:2407.21565

    2024 arXiv

  25. [33]

    , Rabault, J

    Vignon, C. , Rabault, J. , Vasanth, J. , Alc \'a ntara- \'A vila, F. , Mortensen, M. & Vinuesa, R. 2023 Effective control of two-dimensional Rayleigh--B \'e nard convection: Invariant multi-agent reinforcement learning is all you need . Phys. Fluids 35 (6), 065146

    2023

  26. [34]

    & Lauga, E

    Vona, M. & Lauga, E. 2021 Stabilizing viscous extensional flows using reinforcement learning . Phys. Rev. E 104 (5), 055108

    2021

  27. [35]

    & Zhang, M

    Xu, D. & Zhang, M. 2023 Reinforcement-learning-based control of convectively unstable flows . J. Fluid Mech. 954 , A37

    2023

  28. [36]

    & Graham, M

    Zeng, K. & Graham, M. D. 2021 Symmetry reduction for deep reinforcement learning active control of chaotic spatiotemporal dynamics . Phys. Rev. E 104 , 014210

    2021

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.