Integral modelling and Reinforcement Learning control of 3D liquid metal coating on a moving substrate
Pith reviewed 2026-05-22 23:42 UTC · model grok-4.3
The pith
Reinforcement learning with PPO finds a control policy that stabilizes 3D liquid metal films on moving substrates by combining gas jets and electromagnetic actuators.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The PPO identified an optimal control law that reduced interface instabilities via a novel mechanism: gas jets push crests, and electromagnets raise troughs via the Lorentz force.
What carries the argument
The extended integral model of the 3D liquid film on a moving substrate, used as the environment for a Proximal Policy Optimization agent that learns policies for pneumatic and electromagnetic actuators.
If this is right
- The combined gas-jet and electromagnetic actuation can suppress instabilities that neither method achieves alone.
- The integral model supplies a computationally tractable environment for training control policies without full three-dimensional CFD.
- The discovered mechanism (jets on crests, Lorentz lift on troughs) supplies a concrete, testable actuation sequence for coating lines.
- The same reinforcement-learning loop can be re-run for different substrate speeds or film thicknesses once the model is accepted.
Where Pith is reading between the lines
- If the model holds, the same PPO training procedure could be repeated for other body-force actuators such as acoustic or thermal sources.
- The approach suggests that industrial coating uniformity might improve by adding real-time height sensors to close the loop with the learned policy.
- A mismatch between model and experiment would most likely appear first in the phase relationship between actuator commands and the resulting film height.
Load-bearing premise
The extended integral model remains accurate when electromagnetic body forces are added, and the simulation environment used for PPO training faithfully represents the real three-dimensional film dynamics and actuator responses.
What would settle it
Running the learned PPO control law on a physical coating line and measuring whether the observed reduction in wave amplitude matches the model's prediction would confirm or refute the central result.
Figures
read the original abstract
Metallic coatings are used to enhance the durability of metal surfaces by protecting them from corrosion. These protective layers are typically deposited in a fluid state via a liquid film. Controlling instabilities in the liquid film is crucial to achieving uniform, high-quality coatings. This study explores the possibility of controlling liquid films on a moving substrate using a combination of gas jets and electromagnetic actuators. To model the 3D liquid film, we extend existing integral models to incorporate the effects of electromagnetic actuators. The control strategy was developed within a reinforcement learning framework, in which the Proximal Policy Optimisation (PPO) algorithm interacts with the liquid film via pneumatic and electromagnetic actuators to optimise a reward function that accounts for instability-wave amplitude through a trial-and-error process. The PPO identified an optimal control law that reduced interface instabilities via a novel mechanism: gas jets push crests, and electromagnets raise troughs via the Lorentz force.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript extends existing integral (depth-averaged) models for three-dimensional liquid films on a moving substrate to incorporate electromagnetic body forces from actuators, then trains a Proximal Policy Optimization (PPO) agent in simulation to discover a combined gas-jet and electromagnetic control policy that reduces interfacial wave amplitude. The learned policy is described as operating via a novel mechanism in which gas jets displace crests while electromagnets lift troughs through the Lorentz force.
Significance. If the extended integral model remains quantitatively accurate once electromagnetic forcing is added and the resulting policy transfers beyond the training simulator, the work would demonstrate a practical route to multi-actuator stabilization of coating flows that is difficult to obtain by classical control design. The combination of a reduced-order forward model with reinforcement learning is a strength when the model assumptions are verified.
major comments (2)
- [Modeling section] Modeling section (extension of the integral equations): the addition of Lorentz-force terms introduces new streamwise and transverse length scales set by actuator geometry. The manuscript must demonstrate that the long-wave and lubrication assumptions used to derive the depth-averaged system remain valid under these forcings; otherwise the PPO training environment does not faithfully represent the target physics and the reported control law is an artifact of the simulator. This is load-bearing for the central claim.
- [Results section] Results section (PPO policy evaluation): no quantitative comparison is supplied between the controlled and uncontrolled film evolution, nor between the integral-model predictions and either full Navier–Stokes simulations or experimental data once electromagnetic actuators are active. Without such metrics the reduction in instability amplitude cannot be assessed for robustness.
minor comments (2)
- The abstract states that the PPO 'identified an optimal control law' but does not specify the state representation, action space, or reward function; these details should be stated explicitly in the main text.
- Figure captions and axis labels should indicate whether the plotted quantities are nondimensionalized and, if so, by which scales.
Simulated Author's Rebuttal
We thank the referee for their constructive comments. We address each major comment point-by-point below, indicating where revisions will be made to strengthen the manuscript.
read point-by-point responses
-
Referee: [Modeling section] Modeling section (extension of the integral equations): the addition of Lorentz-force terms introduces new streamwise and transverse length scales set by actuator geometry. The manuscript must demonstrate that the long-wave and lubrication assumptions used to derive the depth-averaged system remain valid under these forcings; otherwise the PPO training environment does not faithfully represent the target physics and the reported control law is an artifact of the simulator. This is load-bearing for the central claim.
Authors: We agree that validating the long-wave and lubrication assumptions under electromagnetic forcing is essential for the credibility of the reduced-order model. In the revised manuscript we will add a new subsection to the modeling section that performs an order-of-magnitude analysis of the additional length scales introduced by the actuator geometry. This analysis will confirm that the chosen actuator spacing and operating parameters keep the flow within the regime where the depth-averaged equations remain valid, thereby supporting the use of the integral model as the PPO training environment. revision: yes
-
Referee: [Results section] Results section (PPO policy evaluation): no quantitative comparison is supplied between the controlled and uncontrolled film evolution, nor between the integral-model predictions and either full Navier–Stokes simulations or experimental data once electromagnetic actuators are active. Without such metrics the reduction in instability amplitude cannot be assessed for robustness.
Authors: We acknowledge that explicit quantitative metrics are required to demonstrate the policy's effectiveness. In the revised manuscript we will add direct comparisons between controlled and uncontrolled cases, including time histories of interface amplitude and statistical measures such as RMS wave height reduction. Direct validation against full Navier–Stokes simulations or experiments with active electromagnetic actuators lies outside the scope of the present work, which focuses on the integral-model-plus-RL framework; we will add a clarifying statement in the discussion section regarding this limitation and the planned direction of future validation studies. revision: partial
Circularity Check
No significant circularity detected
full rationale
The paper extends prior integral models by adding electromagnetic body-force terms, then trains a PPO agent inside the resulting simulation to discover a control policy. This is a standard model-based optimization workflow; the discovered policy is an output of the RL process rather than a quantity defined in terms of itself or a fitted parameter relabeled as a prediction. No load-bearing self-citation chains, uniqueness theorems imported from the same authors, or ansatzes smuggled via citation are present in the derivation. The central result therefore remains independent of its own inputs.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
obtained consistent results for a moving substrate at δ= 73.9 andRe= 478, and Barreiro-Villaverdeet al.[23] confirmed the same boundary-layer structure under jet- wiping conditions atδ≈76. In the present configuration, the reduced Reynolds number reachesδ= 266, which ex- ceeds the asymptotic range but remains within one order of magnitude of previously va...
work page 2048
-
[2]
Control with single gas-jets The first case involves a single gas-jet actuator with a prescribed harmonic control function, and the agent is free to take any action at any time. Figure 9 shows the learning curves for the case with a single 2D gas jet, comparing the harmonic control func- tion (green curve with squares) and the nonharmonic control function...
-
[3]
Control with single electromagnets Turning to the control with a single electromagnetic actuator, Figure 13 shows the learning curve (continu- ous blue line with circles and continuous green line with squares), the mean value (red dashed line) and the stan- dard deviation (yellow shaded area) of the reward in the 0.00 0.01 0.02 0.03 t[s] 0.0 0.2 0.4 0.6 0...
-
[4]
The gas jet pushes the crests, whereas the electromagnets raise the valleys
Control with gas jets and electromagnets In the previous test cases, we found that optimal con- trol actions for the gas jet and electromagnetic actua- tors are complementary. The gas jet pushes the crests, whereas the electromagnets raise the valleys. In this case, we test the control of undulations with both actuators op- erating in tandem. Additionally...
-
[5]
N. S. Lalli and A. Giusti, Thin film modelling of magnetic soap films, Journal of Fluid Mechanics986, A7 (2024)
work page 2024
- [6]
-
[7]
F. Pino, B. Scheid, and M. A. Mendez, Multi-objective optimization of the magnetic wiping process in dip- coating, Journal of Engineering Mathematics150, 20 (2025)
work page 2025
-
[8]
A. F. Lehman, Fluid flow control in continuous casting using various configurations of static magnetic fields, in International Symposium on Electromagnetic Processing of Materials, 1994, Nagoya, ISIJ(1994)
work page 1994
-
[9]
L. E. Zakharov, Magnetic propulsion of intense lithium streams in a tokamak magnetic field, Physical review let- ters90, 045001 (2003)
work page 2003
- [10]
- [11]
-
[12]
D. Lunz and P. Howell, Flow of a thin liquid-metal film in a toroidal magnetic field, Journal of Fluid Mechanics 867, 835 (2019)
work page 2019
-
[13]
S. Smolentsev, T. Rhodes, Y. Jiang, P. Huang, and C. Kessel, Status and progress of liquid metal thermoflu- ids modeling for the us fusion nuclear science facility, Fusion Science and Technology77, 745 (2021)
work page 2021
-
[14]
R. Munipalli, S. Shankar, M. Ni, and N. Morley, Devel- opment of a 3-d incompressible free surface mhd com- putational environment for arbitrary geometries: Himag, DOE SBIR phase-ii final report (2003)
work page 2003
-
[15]
S. Smolentsev and M. Abdou, Open-surface mhd flow over a curved wall in the 3-d thin-shear-layer approxima- tion, Applied mathematical modelling29, 215 (2005). 25
work page 2005
- [16]
-
[17]
V. Y. Shkadov, Wave-flow theory for a thin viscous liquid layer, Fluid Dynamics3, 12 (1968)
work page 1968
-
[18]
Kapitza, Wave flow of thin fluid layers of liquids, Sov
S. Kapitza, Wave flow of thin fluid layers of liquids, Sov. Phys., J. Exp. Theor. Phys19, 105 (1949)
work page 1949
-
[19]
M. A. Mendez, A. Gosset, B. Scheid, M. Balabane, and J.-M. Buchlin, Dynamics of the jet wiping process via integral models, Journal of Fluid Mechanics911, A47 (2021)
work page 2021
-
[20]
T. Ivanova, F. Pino, B. Scheid, and M. A. Mendez, Evolu- tion of waves in liquid films on moving substrates, Physics of Fluids35(2023)
work page 2023
-
[21]
D. Barreiro-Villaverde, A. Gosset, and M. A. Mendez, On the dynamics of jet wiping: Numerical simulations and modal analysis, Physics of Fluids33, 062114 (2021)
work page 2021
-
[22]
Buchlin, Modelling of gas jet wiping, Thin liquid films and coating processes (1997)
J.-M. Buchlin, Modelling of gas jet wiping, Thin liquid films and coating processes (1997)
work page 1997
- [23]
-
[24]
M. A. Mendez, K. Myrillas, A. Gosset, and J. Buchlin, A research methodology to study jet wiping processes, in11th European Coating Symposium(Eindhoven, The Netherlands, 2015)
work page 2015
- [25]
-
[26]
D. Barreiro-Villaverde, A. Gosset, M. Lema, and M. A. Mendez, Damping of three-dimensional waves on coating films dragged by moving substrates, Physics of Fluids35 (2023)
work page 2023
-
[27]
D. Barreiro-Villaverde, A. Gosset, M. Lema, and M. A. Mendez, On the coupling instability of a gas jet imping- ing on a liquid film, Journal of Fluid Mechanics992, 10.1017/jfm.2024.553 (2024)
-
[28]
A. W. Wray, R. Cimpeanu, and S. N. Gomes, Electro- static control of the navier-stokes equations for thin films, Physical Review Fluids7, L122001 (2022)
work page 2022
-
[29]
A. B. Thompson, S. N. Gomes, G. A. Pavliotis, and D. T. Papageorgiou, Stabilising falling liquid film flows using feedback control, Physics of Fluids28, 012107 (2016)
work page 2016
-
[30]
Passive and active electromagnetic stabilization of free-surface liquid metal flows
S. Mirhoseini, R. Diaz-Pacheco, and F. Volpe, Passive and active electromagnetic stabilization of free-surface liquid metal flows, arXiv preprint arXiv:1702.01040 (2017)
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[31]
S. Mirhoseini and F. Volpe, Resistive sensor and elec- tromagnetic actuator for feedback stabilization of liquid metal walls in fusion reactors, Plasma Physics and Con- trolled Fusion58, 124005 (2016)
work page 2016
-
[32]
O. A. Holroyd, R. Cimpeanu, and S. N. Gomes, Linear quadratic regulation control for falling liquid films, SIAM Journal on Applied Mathematics84, 940 (2024)
work page 2024
-
[33]
E. Boujo and M. Sellier, Pancake making and surface coating: optimal control of a gravity-driven liquid film, Physical Review Fluids4, 064802 (2019)
work page 2019
-
[34]
R. J. Tomlin, S. N. Gomes, G. A. Pavliotis, and D. T. Papageorgiou, Optimal control of thin liquid films and transverse mode effects, SIAM Journal on Applied Dy- namical Systems18, 117 (2019)
work page 2019
-
[35]
J. Viquerat, P. Meliga, A. Larcher, and E. Hachem, A re- view on deep reinforcement learning for fluid mechanics: An update, Physics of Fluids34(2022)
work page 2022
-
[36]
V. T. Vu, Q. H. Tran, T. L. Pham, and P. N. Dao, Online actor-critic reinforcement learning control for uncertain surface vessel systems with external disturbances, Inter- national Journal of Control, Automation and Systems 20, 1029 (2022)
work page 2022
- [37]
-
[38]
M. Ran, J. Li, and L. Xie, Reinforcement-learning-based disturbance rejection control for uncertain nonlinear sys- tems, IEEE Transactions on Cybernetics52, 9621 (2021)
work page 2021
-
[39]
F. Pino, L. Schena, J. Rabault, and M. A. Mendez, Com- parative analysis of machine learning methods for active flow control, Journal of Fluid Mechanics958, A39 (2023)
work page 2023
- [40]
-
[41]
Proximal Policy Optimization Algorithms
J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, Proximal policy optimization algorithms, arXiv preprint arXiv:1707.06347 (2017)
work page internal anchor Pith review Pith/arXiv arXiv 2017
-
[42]
B. Giordanengo, A. B. Moussa, A. Makradi, H. Chaaba, and J. Gasser, New precise determination of the high temperature unusual temperature dependent ther- mopower of liquid divalent cadmium and zinc, Journal of Physics: Condensed Matter12, 3595 (2000)
work page 2000
-
[43]
A. Gosset and J.-M. Buchlin, Jet wiping in hot-dip galva- nization, Journal of Fluids Engineering129, 466 (2006)
work page 2006
- [44]
-
[45]
S. Kalliadasis, C. Ruyer-Quil, B. Scheid, and M. G. Ve- larde,Falling liquid films, Vol. 176 (Springer Science & Business Media, 2011)
work page 2011
- [46]
-
[47]
P. L. Kapitza and D. t. Haar, Collected papers of pl kapitza, (No Title) (1967)
work page 1967
-
[48]
V. Y. Shkadov, Wave flow regimes of a thin layer of viscous fluid subject to gravity, Fluid Dynamics2, 29 (1967)
work page 1967
-
[49]
S. Beltaos and N. Rajaratnam, Impinging circular tur- bulent jets, Journal of the hydraulics division100, 1313 (1974)
work page 1974
-
[50]
D. Lacanette, A. Gosset, S. Vincent, J.-M. Buchlin, and ´E. Arquis, Macroscopic analysis of gas-jet wiping: Nu- merical simulation and experimental approach, physics of Fluids18, 042103 (2006)
work page 2006
-
[51]
S. Hampton, R. Lane, R. Hedlof, R. Phillips, and C. A. Ordonez, Closed-form expressions for the magnetic fields of rectangular and circular finite-length solenoids and current loops, AIP Advances10(2020)
work page 2020
-
[52]
D. Dutykh, A brief introduction to pseudo-spectral meth- ods: application to diffusion problems, arXiv preprint 26 arXiv:1606.05432 (2016)
- [53]
- [54]
-
[55]
OpenAI, Proximal policy optimization — spinning up documentation,https://spinningup.openai.com/en/ latest/algorithms/ppo.html(2025), accessed: 2025- 01-21
work page 2025
- [56]
- [57]
- [58]
-
[59]
B. V. Derjaguin, On the thickness of the liquid film adher- ing to the walls of a vessel after emptying, Acta Physic- ochim. URSS20, 349 (1943)
work page 1943
-
[60]
F. Pino, M. A. Mendez, and B. Scheid, Linear stability analysis of a vertical liquid film over a moving substrate, Journal of Fluid Mechanics1000, A57 (2024)
work page 2024
-
[61]
P. A. Davidson,An Introduction to Magnetohydrodynam- ics, Cambridge Texts in Applied Mathematics (Cam- bridge University Press, 2001)
work page 2001
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.