{"id":"9155da6e-d462-425b-b8b4-877d17bb35af","arxiv_id":"2502.05577","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Online reinforcement learning controllers reach a simulated lift coefficient near 1.62 within 2.5 seconds for plasma-actuated flow separation control, while adaptive MPC stalls at 1.60.","lead":"Plasma-actuator flow control on a simulated airfoil near stall is compared between a model-based controller and three online reinforcement learning controllers. The learning controllers reach the target lift faster and at a slightly higher value, but the best actuation frequency is not clearly identifiable from the lift signal.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The RL-over-MPC conclusion is confounded: MPC is tasked with tracking an infeasible 1.62 set-point while RL maximizes mean Cl, so the comparison does not isolate algorithm performance.","rationale":"The most load-bearing premise is not that the CFD is perfect, but that the comparison isolates controller capability. The paper's key evidence for MPC's limitation is a single set-point (1.62) that lies above the mean lift RL achieves; MPC's tracking cost (Eq. 10) and RL's reward-maximization objective are different tasks. A tracking controller with an infeasible reference will always 'struggle' in the way described, so the observed failure is largely predetermined and does not warrant the conclusion that RL outperforms MPC. The reader's CFD-fidelity concern is real and would invalidate both methods equally in absolute terms, but it is secondary to this internal confound: even accepting the simulation, the paper has not shown a fair MPC-versus-RL comparison. The concrete test removes the confound by setting MPC's reference to the RL-achieved mean (1.619) and comparing the same metric (mean Cl and fluctuation). If MPC also stabilizes at 1.619, the headline conclusion needs substantial revision; if it cannot, the RL-over-MPC claim is supported. Therefore the paper should remain conditional and require the fair comparison or a clear statement that MPC is not being compared on the same objective.","tokens_in":20517,"tokens_out":6928,"duration_ms":70172,"concrete_test":"Run the adaptive MPC of Section 4.1 on the same CFD case with reference w=1.619 (the mean lift achieved by TDRL/DQL), using the same tuning from Table 1, and report the time-averaged Cl and its fluctuation over the last 2 s. If the mean Cl is within 0.005 of 1.619 and the lift fluctuations are no larger than in the RL runs (Figs. 12-14), the claimed RL superiority is not established. As a second check, quantify MPC's mean Cl for the w=1.62 run; if it already equals roughly 1.619, the 'struggled to reach higher target' statement is misleading.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's headline claim (abstract; Conclusions) is that RL methods outperform adaptive MPC for dual-point plasma separation control. But the two controller classes are evaluated under different objectives. Adaptive MPC minimizes Eq. (10), a tracking cost with a prescribed reference w(t); in Section 5.2 the reference is set to Cl=1.62, and the paper reports only that MPC 'could not stabilize' this set-point and 'oscillates below the set-point.' No time-averaged Cl for the MPC 1.62 case is reported. The RL methods instead maximize a Cl-based reward, and their success metric is a time-averaged mean of 1.619 (Sections 5.3-5.5) — a value below the MPC reference of 1.62. Thus the comparison may reduce to the unsurprising fact that a tracking controller with an infeasible reference fails, while a maximizing controller returns the maximum feasible mean. It does not establish that MPC cannot reach mean lift 1.619, since MPC was never asked to track 1.619 or to maximize mean Cl. The authors' own Section 5.5 notes that 100 and 200 Hz produce near-identical mean lift, so the 'optimal frequency' language is additionally underdetermined. This confound, not the CFD fidelity question alone, is the most load-bearing threat to the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a numerical study of closed-loop active flow control for a NACA 4412 airfoil near static stall (α=15°, Re=4×10^5) using dual-point DBD plasma actuators. Four controllers are compared: adaptive MPC (a generalized predictive controller with recursive least-squares identification), temporal-difference RL, deep Q-learning, and DQL with an LSTM layer. The uncontrolled baseline is validated against external wind-tunnel data (Cp at 14° and trailing-edge wake profiles). The authors report that adaptive MPC tracks a target Cl=1.60 at ~110 Hz but cannot stabilize Cl=1.62, whereas the RL methods reach mean Cl≈1.619 in under 2.5 s at 100 or 200 Hz, and they conclude that RL outperforms MPC for this problem.","tokens_in":20750,"tokens_out":5725,"duration_ms":51872,"significance":"If the central comparison were established, the paper would be a useful contribution: it is among the first to compare MPC with online RL methods for active flow separation control using dual-point DBD plasma actuation, and the baseline validation against external wind-tunnel data is a strength. However, the current evidence does not support the headline claim of RL superiority. The comparison is confounded by different controller objectives, the 'optimal frequency' result is underdetermined by the near-identical means at 100 and 200 Hz, and the controlled-flow simulations rest on a single unverified 2D RANS-SAS setup. The paper is of interest to the active flow control community, but the quantitative comparisons should be regarded as preliminary rather than conclusive.","major_comments":[{"comment":"The headline claim that RL methods outperform adaptive MPC is confounded by asymmetric objectives. The MPC cost in Eq. (10) minimizes tracking error to a reference w(t); in Section 5.2 the Cl=1.62 case is explicitly called 'near the physical limit', and the paper reports only that MPC 'could not stabilize' this set-point, with no time-averaged Cl reported for that run. The RL methods instead maximize a Cl-based reward, and their success metric is a mean Cl of 1.619 (Sections 5.3–5.5), which is below the MPC reference of 1.62. The observed outcome (a tracking controller with an infeasible reference fails, while a maximizing controller returns the maximum feasible mean) does not establish that MPC cannot reach mean Cl=1.619. To support the claim, the authors should either run MPC with the same objective (e.g., tracking 1.619 or maximizing mean Cl) or report the time-averaged Cl for the MPC 1.62 case.","section":"§5.2–§5.5, Eq. (10), abstract/findings"},{"comment":"The controlled results depend entirely on a single 2D RANS-SAS setup with the Shyy plasma body-force model, but the validation is limited to the uncontrolled baseline: Cp at 14° (Fig. 7) and trailing-edge wake profiles (Fig. 8). No grid-convergence, time-step sensitivity, turbulence-model comparison, or three-dimensionality check is provided for the near-stall 15° separated flow used in all controlled cases (Sections 3.3–3.4; 5.2–5.5). The paper itself concedes in the Conclusions that 'future work could improve turbulence modeling'. Because the central quantitative claims (mean Cl=1.619 at 100/200 Hz, MPC's failure at 1.62) are grounded in this unverified simulation environment, the authors should add at least a grid-refinement and time-step sensitivity study, and preferably a turbulence-model comparison, before drawing conclusions about controller performance at these lift levels.","section":"§3.1–§3.4, §5.1"},{"comment":"The action-space description is internally inconsistent. The text states that the discrete action space lies 'within the superharmonic frequencies of wake, specifically from F+ = 1 to F+ = 6' and that 'frequencies beyond this range do not elicit an effective fluid response' (citing [25]), yet Table 2 specifies Actions 0:50:400. With c=0.4 m and U∞=14.6 m/s, F+=f·c/U∞, so the action set includes 250–400 Hz, corresponding to F+ ≈ 6.85–10.96, which the text itself declares ineffective. The RL agents are therefore allowed to choose frequencies the authors state are ineffective. Either restrict the action set to F+ ≤ 6 (≈219 Hz) or justify the inclusion of 250–400 Hz; as written, the 'optimal frequency' search includes known-irrelevant actions.","section":"§4 (paragraph before §4.1) and Table 2"},{"comment":"The claim that RL methods 'effectively optimized excitation frequencies' is underdetermined by the authors' own data. Section 5.5 states that the mean Cl at 100 Hz and 200 Hz is 'nearly identical', and Fig. 15 shows the two duty-cycle traces produce almost the same average. The converged frequency (TDRL and signal-processing DQL select 200 Hz; DQL selects 100 Hz) therefore reflects initial exploration and reward-landscape details rather than a distinct physical optimum. The authors should quantify the Cl difference between 100 and 200 Hz (with a tolerance or statistical test), or else weaken the 'optimal frequency' wording to 'a selected frequency among nearly equivalent options'.","section":"§5.3–§5.5, Fig. 15"},{"comment":"Each RL method is evaluated with a single stochastic run; no seeds, multiple runs, or confidence intervals are reported, and the hyperparameters were chosen 'after multiple initial runs' (Section 4). For a comparison of learning algorithms, single-run outcomes are insufficient to distinguish method performance from run-to-run variance. The authors should report statistics over several independent runs, or at minimum demonstrate insensitivity to exploration seed and to the chosen hyperparameters, before claiming that one RL variant converges faster or more reliably than another.","section":"§4.2–§4.4, §5.3–§5.5"}],"minor_comments":[{"comment":"There are several typographical errors: 'desicion makings' (Sections 5.4 and 5.5), and 'Shy et al.' instead of 'Shyy et al.' (Sections 3.2 and 3.3). These should be corrected.","section":"Throughout"},{"comment":"The lift coefficient notation is inconsistent: 'Cl' is used in most sections but 'CL' appears in the Conclusions. Please standardize to a single notation.","section":"Throughout"},{"comment":"The definition of 'stabilization time' used in Fig. 16 is not given. Please specify the criterion (e.g., time when the moving-average Cl remains within a tolerance band), since the comparison across methods depends on this metric.","section":"§5.6, Fig. 16"},{"comment":"For the MPC Cl=1.60 case, the paper reports that the lift coefficient 'stabilizes' at 2.8 s, but it is unclear whether this refers to the instantaneous value or the moving average shown in Fig. 10. Please clarify what quantity is being used to define stabilization.","section":"§5.2"},{"comment":"The normalized frequency F+ is defined as F+ = f·c/U∞, but U∞ is not defined until Section 3.3 (14.6 m/s). Please define U∞ at first use and state the resulting F+ values for the action set.","section":"§3.2"}],"recommendation":"major_revision","confidential_remarks":"The manuscript fits the journal's scope, but the confounded RL-vs-MPC comparison is likely to draw strong criticism from referees. The authors should be asked to run MPC with the same objective as the RL methods or to report a fair time-averaged comparison, and to add basic verification studies for the controlled-flow CFD setup. The paper's contribution is potentially useful, but the central claim is not yet supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is a clearly written, mostly reproducible 2D CFD case study comparing adaptive MPC (GPC with RLS identification) against TDRL, DQL, and LSTM-DQL for dual-point DBD plasma separation control on a NACA 4412 near stall. The headline claim that RL outperforms MPC does not hold as stated, because the two controller classes are given different jobs. MPC is set up to track a fixed reference; RL is set up to maximize mean Cl. Asking MPC to track 1.62 when the actuator configuration can only sustain about 1.619, then reporting that it struggles, tells you little about which algorithm is better at maximizing lift. The authors even acknowledge the objective difference in Section 5.2, but the abstract and conclusions still present the comparison as a fair win for RL.\n\nWhat is genuinely useful: the baseline validation against external Cp and wake data is real evidence, the algorithm descriptions are specific enough to reimplement, and the honest observation that 100 Hz and 200 Hz give nearly identical mean Cl is a good sanity check that most papers would gloss over. The specific combination of online (no offline pre-training) RL with dual-point actuation, plus the side-by-side with adaptive GPC, is a reasonable data point for the AFC controller-selection literature, even if each ingredient is standard.\n\nSoft spots, in order of importance. First, the confounded comparison: to draw the intended conclusion, run MPC with the same objective, e.g., a set-point of 1.619 or a reward that punishes deviation from the best achievable mean, and give it the same frequency range and settling time. Second, the controlled flow evidence is thin: only the uncontrolled baseline is checked against experiment; the controlled results rest on a single 2D RANS-SAS run with no grid-convergence, time-step, turbulence-model, or 3D sensitivity study and no uncertainty quantification. That is not unusual in this niche, but the limitations should be stated rather than implied. Third, calling either 100 or 200 Hz the \"optimal frequency\" overstates what the data show, since the mean lift values are effectively indistinguishable.\n\nWho should read it: people choosing between MPC and RL for plasma-actuated separation control, and anyone who wants a concrete example of how controller comparisons can be confounded by different objectives. Worth a serious referee, but only with substantial revision: reframe the central claim, add a fair MPC variant, and either add sensitivity analysis or soften the physical claims.","headline":"Competent 2D CFD case study whose central RL-beats-MPC claim is not supported as stated, because MPC is asked to track an infeasible set-point while RL is asked to maximize mean lift.","tokens_in":21361,"tokens_out":2429,"would_cite":false,"duration_ms":25974,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"In a near-stall airfoil simulation, reinforcement learning controllers drive dual-point plasma actuators to a lift coefficient of about 1.62 within 2.5 seconds, while adaptive model predictive control stops at 1.60.","keywords":["active flow control","reinforcement learning","model predictive control","DBD plasma actuator","flow separation control","NACA 4412","dual-point excitation","near-stall aerodynamics"],"falsifier":"Repeat the dual-point DBD control scenario at $Re=4\\times10^5$ and 15 degrees in a three-dimensional wall-resolved large-eddy simulation or a wind-tunnel experiment; the central ranking would be refuted if 100 or 200 Hz excitation does not raise mean lift to about 1.62, or if adaptive MPC can hold 1.62 stably.","tokens_in":20263,"feed_emoji":"⚡","tokens_out":9285,"duration_ms":79173,"temperature":0.7,"pith_summary":"This paper asks which closed-loop controller can better hold a NACA 4412 airfoil near stall under dual-point plasma actuation: adaptive model predictive control or reinforcement learning trained entirely online. In two-dimensional simulations at Reynolds number $4\\times 10^5$ and 15 degrees angle of attack, all three RL variants—temporal-difference Q-learning, deep Q-learning, and deep Q-learning with an LSTM signal-processing layer—found an excitation frequency that raised mean lift to about $C_l = 1.619$ within roughly 2.5 seconds, choosing 100 or 200 Hz. Adaptive MPC reached its set-point of $C_l = 1.60$ at about 110 Hz but could not stabilize $C_l = 1.62$, a value near the actuator's physical limit. The paper's contribution is a direct RL-versus-MPC comparison in one online framework, showing that model-free learning adapts better than model-based tracking in a highly nonlinear, separated-flow regime.","feed_headline":"RL holds airfoil lift at 1.62 where MPC fails","feed_subtitle":"In a near-stall simulation, online RL tunes plasma actuators to 100 or 200 Hz and stabilizes lift; adaptive MPC stops at 1.60.","key_machinery":"The carrying mechanism is the dual-point excitation arrangement of DBD plasma actuators combined with an online control loop. Two plasma zones are placed in the flow: one on the suction side at $x/c = 0.35$, just upstream of the baseline separation point near $0.55c$, and one at the trailing edge on the pressure side at $x/c = 0.99$; both shear layers are excited simultaneously, exploiting their interaction. The plasma region follows a linear electric-field-decay body-force model modulated at a 50% duty cycle with excitation frequencies between 0 and 400 Hz. The controllers read the mean lift coefficient over a short window and output a frequency: adaptive MPC fits a linear ARIMAX model online by recursive least squares and solves a generalized predictive control cost, while the RL agents use an $\\epsilon$-greedy Q-learning update over the discrete action grid 0:50:400 Hz, with reward emphasizing improvement over the baseline $C_l = 1.44$. The mechanism that makes dual-point excitation effective is periodic forcing that locks onto the separated shear layer, forming coherent vortices and entraining high-momentum fluid, which shrinks the separation bubble and raises lift.","core_discovery":"The paper's central claim is that reinforcement learning controllers identify and hold a better operating point than adaptive MPC for the same dual-point dielectric-barrier-discharge plasma actuator system on a NACA 4412 airfoil at 15 degrees and $Re = 4\\times10^5$. TDRL, DQL, and DQL with signal processing each converge to a mean lift coefficient of approximately 1.619, with TDRL and the signal-processing variant settling on 200 Hz and DQL on 100 Hz; the two frequencies produce nearly equal mean lift through a 50% duty-cycle averaging effect. Adaptive MPC, a generalized predictive controller with recursive least-squares identification, reaches $C_l = 1.60$ at about 110 Hz ($F^+\\approx 3$) but becomes unstable trying to hold $C_l = 1.62$, which lies at the physical limit of the dual-actuator configuration. The paper attributes the RL advantage to online, reward-driven exploration of the discrete frequency space and attributes the improved aerodynamics to shear-layer instability amplification, lock-on, and enhanced mixing that reduce the suction-side separation zone and increase the pressure differential across the airfoil.","pith_inferences":["An implication the paper leaves implicit is that the 100 Hz versus 200 Hz tie points to a plateau in the lift-versus-frequency landscape; a controller that adds a small dithering sweep could map that plateau and reduce ambiguity.","Because the results come from a 2D RANS-SAS model with a simplified plasma forcing model, a natural test is whether the same frequency preference survives three-dimensional wall-resolved simulations or wind-tunnel experiments, where spanwise instabilities may shift the optimum.","The same online state-reward loop could tune other actuation parameters, such as duty cycle, voltage amplitude, or the phase offset between the two plasma zones, since the controller only needs a scalar performance signal.","For MPC, the failure near $C_l=1.62$ is a set-point-tracking saturation problem; switching to RL or augmenting the identified linear model with a saturation or disturbance model could extend adaptive MPC's useful range."],"forward_implications":["If the comparison holds, online RL can control near-stall separation without a pre-trained dataset or an explicit plant model, reaching $C_l\\approx 1.62$ within about 2.5 seconds of simulated time.","Adaptive MPC remains suitable for moderate set-points: it stabilizes $C_l = 1.60$ at about 110 Hz, which is close to the 100 Hz that DQL selects, so both approaches converge on $F^+\\approx 3$ as an effective excitation regime.","The near-identical mean lift at 100 and 200 Hz implies that mean $C_l$ alone cannot distinguish these two operating points; richer feedback, such as lift fluctuation amplitude or spectral content, would be needed to break the tie.","More complex RL architectures make more decisions yet stabilize the lift in less simulated time, indicating that sequence-aware state representations accelerate convergence.","Since the dual-actuator configuration saturates near $C_l = 1.62$, further lift improvements would require changing the actuation layout or authority rather than only improving the controller."],"supporting_citations":[{"why":"It supplies the dual-point excitation method and the placement rationale for the two actuators on the airfoil.","marker":"[36]"},{"why":"It provides the linear-decay plasma body-force model and the actuator parameters used in the simulations.","marker":"[52]"},{"why":"It supplies the scale-adaptive turbulence model that lets the solver capture a wider range of turbulent scales.","marker":"[51]"},{"why":"It supplies the experimental pressure-coefficient data used to validate the uncontrolled baseline at 14 degrees.","marker":"[65]"},{"why":"It supplies the experimental and numerical wake-velocity data used for the trailing-edge validation.","marker":"[66]"},{"why":"It provides the generalized predictive control algorithm on which the adaptive MPC implementation is based.","marker":"[61]"},{"why":"It demonstrates deep Q-network closed-loop separation control and motivates the discrete frequency range $F^+=1$ to $6$ used for actions.","marker":"[25]"},{"why":"It provides the deep Q-learning algorithm and network-training approach adapted for the DQL variant.","marker":"[63]"},{"why":"It provides the LSTM layer used in the signal-processing-enhanced DQL variant.","marker":"[64]"}],"fun_headline_variants":["RL beats MPC for airfoil lift: 1.62 vs 1.60","RL reaches Cl=1.62 on airfoil, MPC maxes at 1.60","RL tunes plasma actuators to beat MPC in airfoil lift","Reinforcement learning outperforms MPC for stalled airfoil control","RL holds airfoil lift at 1.62, MPC can't reach plateau"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the two-dimensional RANS-SAS simulation with the simplified plasma body-force model faithfully reproduces the near-stall separated-flow dynamics the controllers exploit; the paper validates this setup only for the uncontrolled baseline, not for the controlled cases.","fun_headline_variants_meta":{"raw":{"variants":["RL beats MPC for airfoil lift: 1.62 vs 1.60","RL reaches Cl=1.62 on airfoil, MPC maxes at 1.60","RL tunes plasma actuators to beat MPC in airfoil lift","Reinforcement learning outperforms MPC for stalled airfoil control","RL holds airfoil lift at 1.62, MPC can't reach plateau"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000694,"raw_usage":{"total_tokens":3195,"prompt_tokens":1059,"completion_tokens":2136,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":675,"completion_tokens_details":{"reasoning_tokens":2040}},"tokens_in":675,"tokens_out":2136,"duration_ms":14942,"temperature":1.0,"reasoning_tokens":2040,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T18:46:40.583717+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Repeat the dual-point DBD control scenario at $Re=4\\times10^5$ and 15 degrees in a three-dimensional wall-resolved large-eddy simulation or a wind-tunnel experiment; the central ranking would be refuted if 100 or 200 Hz excitation does not raise mean lift to about 1.62, or if adaptive MPC can hold 1.62 stably.","supporting_citations":[{"cited_title":"Ebrahimi, M","cited_arxiv_id":null,"evidence_quote":"It supplies the dual-point excitation method and the placement rationale for the two actuators on the airfoil."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the linear-decay plasma body-force model and the actuator parameters used in the simulations."},{"cited_title":"Menter, Y","cited_arxiv_id":null,"evidence_quote":"It supplies the scale-adaptive turbulence model that lets the solver capture a wider range of turbulent scales."},{"cited_title":"Mallor, C","cited_arxiv_id":null,"evidence_quote":"It supplies the experimental pressure-coefficient data used to validate the uncontrolled baseline at 14 degrees."},{"cited_title":"Tabatabaei, M","cited_arxiv_id":null,"evidence_quote":"It supplies the experimental and numerical wake-velocity data used for the trailing-edge validation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"It provides the generalized predictive control algorithm on which the adaptive MPC implementation is based."},{"cited_title":"Shimomura, S","cited_arxiv_id":null,"evidence_quote":"It demonstrates deep Q-network closed-loop separation control and motivates the discrete frequency range $F^+=1$ to $6$ used for actions."},{"cited_title":"Schmidhuber, S","cited_arxiv_id":null,"evidence_quote":"It provides the LSTM layer used in the signal-processing-enhanced DQL variant."}],"review_version":1}