{"id":"f6946aa6-6ee8-4c64-a0b3-13e66cb6f164","arxiv_id":"2506.20554","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"A closed-loop RL controller dynamically yaws turbines in LES and raises wind farm power by 4.30%, nearly doubling the 2.19% gain of static Bayesian-optimized yaw angles.","lead":"A reinforcement learning controller steered three wind turbines in a high-fidelity simulation, boosting simulated farm power by 4.30% over greedy operation versus a 2.19% gain from static yaw optimization. The work suggests that sensor-driven, closed-loop wake steering could nearly double the benefit of current static wake-steering methods.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The replay test in §2.3 does not establish that the 4.30% gain requires closed-loop feedback; an open-loop dynamic yaw schedule may capture much of the advantage.","rationale":"Good-faith reading: the paper is a serious simulation study; the LES+RL coupling is a real engineering contribution and the internal statistics (224 episodes, confidence intervals) are reported. The numerical result 4.30% over greedy is plausible conditional on the actuator-disk LES. The reader's weakest assumption (LES fidelity) is real but is a standard limitation of any simulation study and is not the place where the paper's specific novel claim is most fragile. The paper's distinct contribution is 'closed-loop collaborative control'; that claim is supported by two pieces of evidence: the replay test and the barycenter correlations. The barycenter correlations show the policy conditions on upstream velocity, but they do not establish that conditioning is necessary for the power gain. The replay test compares the closed-loop policy against a single action trajectory, which is not the right open-loop baseline. If the policy were stochastic at evaluation, the replay test would fail even for an open-loop policy; if deterministic, it still only tests portability of one trajectory, not optimal open-loop dynamic control. The paper never trains or evaluates a fixed dynamic yaw schedule, despite its own switching analysis suggesting exactly such a schedule as a natural baseline. In fact, the Discussion's first future-work item proposes constraining the controller to switch between static optima, which is an open-loop dynamic policy; comparing against that would directly quantify the value of feedback. Therefore the title-level claim is conditional on an ablation that is missing. The verdict should remain CONDITIONAL as the reader recommended, but the condition should explicitly include an observation-ablated or optimal-open-loop baseline; without it, the central mechanism claim is unverified.","tokens_in":17612,"tokens_out":13823,"duration_ms":156910,"concrete_test":"Train the same SAC agent on the same LES environments with the observation vector replaced by a constant (or by observations randomly shuffled across episodes) so that sensors carry no information, keeping all hyperparameters, episode length, and evaluation protocol identical. If this observation-ablated policy achieves a mean gain close to 4.30% on the 224-episode protocol, the closed-loop claim fails; if it falls to roughly the static-BO level (~2.2%), the replay test's conclusion is supported. A cheaper intermediate check: average the yaw action trajectories over the 224 evaluation episodes to form a single open-loop schedule, replay this schedule on the 16 independent inflow environments, and compare its mean power gain to 4.30% and 2.19%. If the averaged schedule recovers more than ~3%, the closed-loop interpretation needs revision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing premise that the RL gain is due to closed-loop feedback rests on the replay test in §2.3. The test saves one action trajectory from the converged policy, replays it on statistically independent inflow realisations, and observes no gain (95% CI [−0.35%, +0.96%]). This is not a sufficient control for open-loop dynamic control. An open-loop policy would not be a single replayed trajectory; it would be a fixed yaw schedule optimised over the training/inflow distribution, for example periodic switching between the two discovered modes ±α_BO at the gust-advection time scale. The paper's own switching analysis (§2.4, Figure 3) shows the controller toggles between approximately +α_RL=(23°,12°,−1°) and −α_RL=(−22°,−8°,7°), and that these modes are close to ±α_BO. A periodic open-loop version of this toggle, phase-locked to nothing but repeated at the relevant time scale, could plausibly recover a substantial fraction of the 2.1 percentage-point advantage over static BO. The replay test only shows that the particular closed-loop action sequence is not portable to different inflow; it does not show that no fixed dynamic sequence could achieve comparable power. The barycenter correlations (Figure 3f–h) demonstrate that the policy uses sensor information, but they do not quantify how much of the power gain is causally attributable to that information. Consequently, the title's 'Closed-Loop Collaborative Control' and the paper's claim to be the first successful dynamic closed-loop RL wake-steering controller are not yet established by the presented evidence.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents an RL-based yaw controller for a three-turbine, in-line wind farm simulated with the Winc3D large-eddy simulation solver. The controller receives upstream velocity-sensor measurements and updates yaw angles every 10 s, and it is trained with the SAC algorithm across 32 parallel LES environments. Evaluated on 224 statistically independent episodes, the RL policy yields a mean power increase of 4.30% relative to greedy operation, compared with 2.19% for a static yaw configuration found by Bayesian optimization. The authors use a closed-loop replay test, cross-correlation analysis, spectral analysis, and flow-field visualizations to argue that the gain arises from closed-loop switching between two near-symmetric yaw modes in response to turbulent gusts.","tokens_in":17943,"tokens_out":5611,"duration_ms":65452,"significance":"If the quantitative claims survive scrutiny, this is a valuable demonstration that RL can be coupled directly to high-fidelity LES for dynamic wind farm control, with a careful statistical evaluation protocol and openly available code. The replay test is a thoughtful attempt to distinguish closed-loop from open-loop behavior, and the independent-inflow evaluation with confidence intervals is a strength. However, two load-bearing aspects require attention: the replay test does not exclude the possibility that a well-chosen open-loop dynamic schedule achieves a large fraction of the gain, and the reported headline improvement is susceptible to selection bias because hyperparameters were chosen by sweeps that used evaluation power as the selection criterion.","major_comments":[{"comment":"The replay test does not establish that the 4.30% gain requires closed-loop feedback. Replaying a single action trajectory from the converged policy on statistically independent inflows shows only that this particular trajectory is not portable to different inflow conditions; it does not compare against an open-loop dynamic controller optimized over the inflow distribution. Concretely, one would need to evaluate a periodic schedule that switches between the two modes α_RL^+ and α_RL^- identified in §2.4 and Figure 3, with frequency and phase swept, or train an RL policy with the same action space but no sensor observations. The authors' own Discussion item 1 suggests that switching between static optima is a plausible alternative design, which further highlights the need for this control. The statement that 'accurate sensor measurements are required' is therefore too strong, and the title's 'Closed-Loop Collaborative Control' claim is not yet fully supported.","section":"§2.3"},{"comment":"The headline 4.30% is likely optimistically biased because the SAC hyperparameters (discount factor, actor/critic learning rate, entropy learning rate, initial entropy multiplier, and number of velocity probes) were selected by sweeps that used evaluation power as the selection criterion. The reported 95% confidence interval [4.10%, 4.49%] treats the policy as a pre-specified estimator rather than as the best of many configurations examined. To place the RL result on equal statistical footing with the BO baseline, the authors should either report the total number of hyperparameter configurations evaluated and compute the final estimate on a holdout set, or apply a multiple-comparisons correction. Without this, the comparison '4.30% versus 2.19%' may overstate the true advantage of the RL approach.","section":"§2.1, Appendix Table 1 and Figure 11"},{"comment":"The bandwidth-limiting study replays low-pass filtered versions of the same yaw trajectories on inlet conditions identical to those used to generate the trajectories. This setup can demonstrate that the learned policy contains a dominant low-frequency component, but it cannot provide an unbiased estimate of the power that a bandwidth-limited closed-loop controller would achieve on independent inflows. The claim that the controller operates at an effective frequency below 3.125×10^-3 Hz and with yaw rates below 0.4°/s is presented as a practical feasibility result; it should be either evaluated on independent inflow conditions or explicitly qualified as a property of the specific replayed trajectory.","section":"§2.5, Figure 4(c)(d)"}],"minor_comments":[{"comment":"The sentence 'SAC is a model-free RL algorithm without policies' appears to contain a typo; presumably the intended meaning is that SAC is an off-policy algorithm or that it does not require on-policy updates.","section":"§4.3.2"},{"comment":"References [18] and [19] are the same paper (Font et al., 2025, Deep reinforcement learning for active flow control in a turbulent separation bubble) and should be merged or renumbered.","section":"References"},{"comment":"The statement that the maximum yaw velocity is reduced 'from 1°/s to below 0.4°/s' is partly by construction, since 1°/s is the imposed action limit; the more informative quantity is the effect of the filter on the resulting power, which is currently only shown for the non-independent replay.","section":"§2.5"},{"comment":"The angle-penalty term is described as 'large', but the chosen values λ=0.5 and κ=26 are not varied in the reported sweeps; a sentence justifying this choice or citing a sensitivity test would be useful.","section":"§4.3.3, Eq. (14)"}],"recommendation":"major_revision","confidential_remarks":"The manuscript reports an impressive simulation result with a serious evaluation protocol, and the two main concerns are addressable within the scope of a revision: adding an open-loop dynamic baseline and de-biasing the hyperparameter selection. I would not recommend rejection, but the central 'closed-loop' attribution and the quantitative headline should be revised to match the evidence."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper does something genuinely new: it trains a SAC yaw controller directly against an LES and shows, on 224 independent evaluation episodes, a 4.30% mean power gain over greedy operation versus 2.19% for static Bayesian optimization. The confidence intervals are tight, the evaluation protocol is sound, and the code is public. That is a real result worth building on.\n\nThe main soft spot is exactly what the stress-test note flags. The replay test in §2.3 shows that a single saved action trajectory fails on independent inflows, but that is not a control for an open-loop dynamic schedule. The paper's own analysis shows the controller toggles between modes close to ±α_BO at a timescale near the gust advection time. A periodic schedule switching between those two static optima could plausibly recover a good fraction of the 2.1-percentage-point margin over static BO, and the paper never tests that. The barycenter correlations show the policy uses sensor information, but they do not quantify how much of the power gain causally requires that information. So the title's \"closed-loop collaborative control\" overstates what the evidence establishes.\n\nAlso worth noting: the hyperparameters were selected by sweeps that used evaluation power (Figure 11), so the headline 4.30% is likely optimistic. The bandwidth-limiting study in §2.5 is a nice practical contribution but omits the numeric power results, making it hard to judge. The actuator-disk LES at D/10 is reasonable for a proof of concept; the authors are appropriately cautious in the Discussion about field transfer.\n\nNone of this is fatal. The paper is internally coherent, the negative replay result is at least consistent with closed-loop behaviour, and the authors have openly listed fatigue, sensing, and generalization as deployment barriers. The switching analysis is thoughtful and suggests a simpler controller design, which the authors themselves point out.\n\nI would send this to peer review. The right path is a major revision that adds an open-loop dynamic baseline (for example, periodic switching between ±α_BO at the relevant timescale) and rewrites the closed-loop claims to match what the evidence actually supports. The paper deserves a serious referee despite the current overstatement.","headline":"A credible first demonstration of RL-based dynamic yaw control in LES, but the replay test does not fully establish the closed-loop claim without an open-loop dynamic baseline.","tokens_in":18516,"tokens_out":2529,"would_cite":false,"duration_ms":32369,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A reinforcement learning controller that yaws wind turbines every 10 seconds raises simulated farm power by 4.30%, nearly doubling the 2.19% gain from static optimal yaw angles.","keywords":["wind farm control","wake steering","reinforcement learning","large-eddy simulation","closed-loop control","yaw control","Bayesian optimization"],"falsifier":"Re-run the same training and evaluation with actuator-line turbine models (which resolve blade forces) instead of actuator disks at the same grid spacing, and compare the RL controller's mean power gain to the static Bayesian optimum's gain; if the closed-loop controller no longer nearly doubles the static gain, the claim that dynamic feedback is the source of the improvement is falsified.","tokens_in":17390,"feed_emoji":"🌬️","tokens_out":9873,"duration_ms":95878,"temperature":0.7,"pith_summary":"This paper argues that wind farm control does not have to be static: a reinforcement learning agent trained directly against a high-fidelity large-eddy simulation of a three-turbine farm learns to yaw each turbine in response to live velocity measurements and raises mean farm power by 4.30% over greedy operation, nearly double the 2.19% gain from the best static yaw angles found by Bayesian optimization. The key claim is that the extra gain is genuinely closed-loop: replaying the controller's recorded yaw sequence on statistically independent inflows produces no gain, which shows the improvement depends on sensing and reacting to the flow rather than executing a fixed pattern. If the result is right, it shifts wind farm optimization from steady-state wake-model tuning toward dynamic, flow-responsive coordination, and the trained controller is cheap enough to evaluate in real time. The authors state this is the first successful demonstration of an efficient dynamic closed-loop reinforcement-learning controller for wind farm control.","feed_headline":"RL closed-loop yaw control lifts wind-farm power 4.3%","feed_subtitle":"A policy that re-yaws turbines every 10 s beats static wake steering by sensing turbulence in real time.","key_machinery":"The load-bearing machinery is the coupling of a high-fidelity large-eddy simulation with actuator disk turbine models to a Soft Actor-Critic (SAC) reinforcement learning policy, an algorithm that adds entropy regularization to balance exploration and exploitation. The policy is a multi-layer perceptron that maps 231 hub-height streamwise velocity sensors plus the previous yaw angles to yaw angular velocities, updating every 10 seconds with a limit of 1 degree per second and a maximum yaw of ±40 degrees. The reward is instantaneous farm power minus a large-angle penalty. The decisive verification device is the action-replay test: executing the recorded yaw sequence in fresh environments with independent inflow removes the power gain, showing the learned control is feedback-driven rather than a fixed open-loop forcing pattern.","core_discovery":"The paper reports that a Soft Actor-Critic RL agent, controlling the yaw of three in-line turbines separated by five rotor diameters, learns to alternate between two near-mirror-image yaw modes, $\\vec{\\alpha}^+_{RL}=(23^\\circ,12^\\circ,-1^\\circ)$ and $\\vec{\\alpha}^-_{RL}=(-22^\\circ,-8^\\circ,7^\\circ)$, which are close to the static Bayesian optimum and its reflection. Cross-correlation of the yaw signals shows the controller has learned the advection delay between turbines, and joint densities with the lateral barycenter of the upstream velocity show it yaws against incoming flow asymmetries, steering high-momentum gusts toward downstream turbines. Across 224 evaluation episodes the RL controller produces a 4.30% mean power gain (95% CI [4.10, 4.49]) versus 2.19% (95% CI [1.98, 2.39]) for static Bayesian optimization. The closed-loop nature is established by an action-replay test: the stored action sequence, replayed on independent inflow conditions, gives no statistically significant gain (95% CI [−0.35, +0.96]). The paper therefore claims that dynamic flow-responsive wake steering, not just optimized steady angles, is what nearly doubles the achievable power increase.","pith_inferences":["Editorial inference: If the switching-between-two-optima mechanism is the true source of the gain, a much simpler hand-designed controller—thresholding on the upstream velocity barycenter to pick between the two static yaw modes—might reproduce most of the 4.30% gain without any neural network; this is directly testable in the same LES environments.","Editorial inference: The paper's own frequency analysis implies the controller is tracking large-scale, slowly varying gusts rather than small-scale turbulence; a plausible extension is that training on a wider ensemble of wind speeds and turbulence intensities would produce a policy whose gain generalizes beyond the single inflow condition studied.","Editorial inference: The authors' suggestion to constrain a dynamic controller to switch among finitely many static optima could be formalized as a hybrid supervisory control problem, reducing both training cost and the risk of mechanically damaging yaw actuators."],"forward_implications":["The trained controller runs in real time at essentially no compute cost, so the expensive supercomputer-based training is a one-time investment; deployment only needs upstream flow sensing and yaw actuation within about 1 degree per second.","The learned strategy deliberately sacrifices first-turbine power to redirect high-momentum gusts toward downstream turbines, implying that greedy per-turbine operation leaves a repeatable power gain of about 4% on the table for this three-turbine configuration.","The controller's dominant actuation frequency ($\\mathrm{St}\\approx 2\\times 10^{-2}$) and reduced yaw rates (below 0.4 degrees per second after filtering) sit within ranges likely achievable by current commercial turbines, so the dynamic strategy is not obviously infeasible mechanically.","Because the replayed action sequence yields no gain, any practical deployment must preserve the closed loop: open-loop or scheduled yaw patterns, even with optimal static angles, forfeit most of the benefit.","The static Bayesian optimum found in this study—upstream turbines yawed more, downstream less—is consistent with the controller's two switching modes, suggesting the dynamic policy simply modulates between known good setpoints based on instantaneous flow asymmetry."],"supporting_citations":[{"why":"Validates the actuator disk turbine model for far-wake flow and power predictions at the grid resolution used here, grounding the simulator's quantitative outputs.","marker":"[58]"},{"why":"Documents and validates the specific actuator disk implementation used for the turbine forces and power calculation in the LES.","marker":"[7]"},{"why":"Supplies the actuator disk and one-dimensional momentum theory and the LES approach for turbine-array boundary layers on which the solver's turbine model is based.","marker":"[11]"},{"why":"Describes the wind-farm LES solver used to generate all training and evaluation environments.","marker":"[15]"},{"why":"Introduces the Soft Actor-Critic algorithm that trains the yaw policy, including the entropy regularization hyperparameters swept in this paper.","marker":"[24]"},{"why":"Recent demonstration that RL outperforms classical open-loop forcing in high-fidelity turbulent flow control, motivating the closed-loop approach and the comparison standard.","marker":"[18]"},{"why":"Prior attempt coupling RL to LES for three turbines, where torque and pitch control failed to yield a dynamic closed-loop strategy; this paper's result is positioned against that gap.","marker":"[39]"},{"why":"Field-scale evidence that collective wind farm operation based on predictive models increases energy production, establishing wake-steering gains that dynamic control aims to exceed.","marker":"[31]"},{"why":"Review of wake management establishing the static yaw-steering strategy (upstream turbines yawed more) that the Bayesian optimization baseline and the RL switching modes follow.","marker":"[28]"},{"why":"LES modeling of a large offshore wind farm with turbine and atmospheric parameters used to set the precursor boundary-layer conditions for the simulations.","marker":"[74]"}],"fun_headline_variants":["Closed-loop RL yaw control nearly doubles wind-farm gain","RL re-yaws turbines in real time to gain 4.3% wind power","RL yaw policy doubles static gain in wind-farm power","RL closed-loop control learns wind gusts, adds 4.3% power"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the simulated turbine model—each rotor represented as a porous disk in a large-eddy simulation with a grid spacing of one-tenth of the rotor diameter—faithfully reproduces the wake recovery and transient power response that occur in real wind farms, since every quantitative claim in the paper comes from this simulator.","fun_headline_variants_meta":{"raw":{"variants":["Closed-loop RL yaw control nearly doubles wind-farm gain","RL re-yaws turbines in real time to gain 4.3% wind power","RL yaw policy doubles static gain in wind-farm power","RL closed-loop control learns wind gusts, adds 4.3% power"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00083,"raw_usage":{"total_tokens":3640,"prompt_tokens":973,"completion_tokens":2667,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":589,"completion_tokens_details":{"reasoning_tokens":2588}},"tokens_in":589,"tokens_out":2667,"duration_ms":21941,"temperature":1.0,"reasoning_tokens":2588,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T22:47:22.404967+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the same training and evaluation with actuator-line turbine models (which resolve blade forces) instead of actuator disks at the same grid spacing, and compare the RL controller's mean power gain to the static Bayesian optimum's gain; if the closed-loop controller no longer nearly doubles the static gain, the claim that dynamic feedback is the source of the improvement is falsified.","supporting_citations":[{"cited_title":"Energies 14(13):3745","cited_arxiv_id":null,"evidence_quote":"Validates the actuator disk turbine model for far-wake flow and power predictions at the grid resolution used here, grounding the simulator's quantitative outputs."},{"cited_title":"Journal of Fluid Mechanics 955:A12","cited_arxiv_id":null,"evidence_quote":"Documents and validates the specific actuator disk implementation used for the turbine forces and power calculation in the LES."},{"cited_title":"Physics of Fluids 22(1):015110","cited_arxiv_id":null,"evidence_quote":"Supplies the actuator disk and one-dimensional momentum theory and the LES approach for turbine-array boundary layers on which the solver's turbine model is based."},{"cited_title":"Wind Energy","cited_arxiv_id":null,"evidence_quote":"Describes the wind-farm LES solver used to generate all training and evaluation environments."},{"cited_title":"In: International conference on machine learning, PMLR, pp 1861–1870","cited_arxiv_id":null,"evidence_quote":"Introduces the Soft Actor-Critic algorithm that trains the yaw policy, including the entropy regularization hyperparameters swept in this paper."},{"cited_title":"Journal of Physics: Conference Series 1934(1):012022","cited_arxiv_id":null,"evidence_quote":"Prior attempt coupling RL to LES for three turbines, where torque and pitch control failed to yield a dynamic closed-loop strategy; this paper's result is positioned against that gap."},{"cited_title":"Nature Energy 7(9):818–827","cited_arxiv_id":null,"evidence_quote":"Field-scale evidence that collective wind farm operation based on predictive models increases energy production, establishing wake-steering gains that dynamic control aims to exceed."},{"cited_title":"Wind Energy 25(2):195–220","cited_arxiv_id":null,"evidence_quote":"Review of wake management establishing the static yaw-steering strategy (upstream turbines yawed more) that the Bayesian optimization baseline and the RL switching modes follow."},{"cited_title":"Renewable Energy 75:945–955","cited_arxiv_id":null,"evidence_quote":"LES modeling of a large offshore wind farm with turbine and atmospheric parameters used to set the precursor boundary-layer conditions for the simulations."}],"review_version":1}