{"id":"d1127d4d-368f-4c20-b6c7-9cd918d0d5bf","arxiv_id":"2411.13194","paper_version":2,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Solver-in-the-loop training yields a stable turbulence shell-model closure that reproduces high-order statistics, with an optimal time-in-loop related to the cutoff-shell eddy turnover time.","lead":"The authors trained a neural network to supply the missing small-scale terms in a simplified model of turbulence, using a training method that lets the network interact with the solver over time. The resulting closure reproduced rare high-order turbulent fluctuations, and the best training horizon tracked the smallest resolved eddy time.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The three-shell locality assumption in Section III is asserted without a convergence study; if the true subgrid closure is nonlocal or has memory, the central reproducibility claim fails.","rationale":"Reader's weakest assumption matches the most load-bearing point. The paper delivers a convincing internal comparison for the specific Nc=14, Re~1e12 setup: flatness up to order 10, scaling exponents, Lagrangian structure functions, flux pdf, and local slopes all track GT, and the model is stable beyond training window. Those results are real evidence, and the absence of code or repeated-seed error bars, while important for reproducibility, does not by itself undermine the physics claim. The optimal-time-in-loop rule (0.41 tau_Nc) is a secondary physical hypothesis; it is derived from a sparse sweep without error bars but the core demonstration does not depend on that exact value. The input-locality assumption is different: it is the architectural precondition for the closure. The paper's one-sentence assertion that three shells are sufficient is not backed by a convergence study, and the dynamics of the unresolved shells do not make it obvious. A deterministic map from three resolved shells to the subgrid flux could be adequate if the unresolved shells are effectively slaved, but that is exactly what needs to be shown. I therefore keep the reader's CONDITIONAL verdict, with the added condition that the locality assumption be tested as described. Since the conditional verdict already flags the locality assumption, no change in verdict is needed.","tokens_in":14168,"tokens_out":5431,"duration_ms":60048,"concrete_test":"Compute from the GT dataset the conditional variance of the subgrid flux Π_Nc (Eq. 15) given the three complex shells (u_{Nc-2},u_{Nc-1},u_{Nc}) using histogram or kNN estimators; if the explained fraction of flux variance is low, no instantaneous deterministic closure with these inputs can be sufficient. Independently, retrain the MLP with input windows of 2, 3, 4, 5, and 6 shells before the cutoff (and with a one-step recurrent memory), keeping Nc=14 and all other hyperparameters fixed, and compare flatness F^(4..10), flux pdf, and local-slope MSE. A clear plateau at three shells plus high explained variance would support the assumption; a material improvement with wider inputs or memory would refute it.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section III states: 'As input to the neural network, we provide the three shells preceding the cutoff, which is sufficient to close the flux locally.' The supporting sentence about fewer/more shells is not accompanied by any quantitative study or figure. The exact unresolved terms in Eqs. (2)-(3) involve only u_{Nc+1} and u_{Nc+2}, but the nontrivial part of the assumption is that those two shells are an instantaneous function of only (u_{Nc-2}, u_{Nc-1}, u_{Nc}). Sabra dynamics for the unresolved shells are autonomous chaotic ODEs coupled to the resolved state; nothing guarantees that their current values (or the resulting flux) are slaved to three neighboring resolved shells. If the conditional distribution of the subgrid flux given the three-shell state has substantial variance, a deterministic MLP with this input cannot represent the true closure, and the reported agreement of high-order moments and the flux pdf, including backscatter, would be a coincidence of this particular cutoff and dataset rather than a general property of solver-in-the-loop training. This assumption is load-bearing because the MLP architecture and the claim of reproducibility at high Reynolds number both rest on it.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper applies a posteriori (solver-in-the-loop) training to a neural-network closure for the Sabra shell model of turbulence, with the cutoff placed at shell Nc=14 inside the inertial range at extremely high Reynolds number. The closure is an MLP that takes the three shells immediately preceding the cutoff as input and outputs the two shells after the cutoff, which are needed to advance the resolved equations. Training unrolls the differentiable reduced solver for msteps time steps and minimizes a velocity MSE against fully resolved ground truth. The paper reports that the learned closure reproduces flatness up to order 10, Eulerian structure functions and anomalous scaling exponents, Lagrangian structure functions, velocity pdfs, and the subgrid flux pdf including backscatter, and that it outperforms an LSTM and a phenomenological closure on local slopes near the cutoff. It further studies the effect of the unrolling time msteps and concludes that the optimal time in the loop is approximately 0.41 of the eddy turnover time of the fastest shell included in the loss, arguing that this is a physical time scale for the method.","tokens_in":14415,"tokens_out":4258,"duration_ms":44471,"significance":"If the results hold, the paper makes a useful contribution by demonstrating that a simple, memoryless MLP closure trained with a differentiable solver can reproduce strongly non-Gaussian, intermittent statistics of high-Reynolds-number shell-model turbulence, and by proposing a heuristic for choosing the unrolling time. The paper includes several strengths: comparisons against ground truth on held-out test time, chunked error bars on the main statistical observables, a stability test beyond the training horizon, a direct comparison with an LSTM closure and a phenomenological closure, and attention to backscatter in the flux pdf. The central empirical demonstration is not circular, since training uses the ground-truth velocity as a target and evaluation is on different initial conditions and time intervals. However, the generalizability of the method rests on a locality assumption that is asserted but not quantitatively tested, and the proposed time-in-loop rule is fitted post hoc to a single configuration, which limits the strength of the conclusions.","major_comments":[{"comment":"The claim that the three shells preceding the cutoff 'is sufficient to close the flux locally' is load-bearing but unsupported. The text states that fewer shells hurt and more do not help, but no figure, table, or quantitative convergence study is provided. Since the MLP maps only (u_{Nc-2}, u_{Nc-1}, u_{Nc}) to (u_{Nc+1}, u_{Nc+2}), the entire method presupposes that the two unresolved shells are an instantaneous function of these three resolved shells. The Sabra dynamics of the unresolved shells are autonomous chaotic ODEs coupled to the resolved state, so memory or nonlocal dependence is not excluded a priori. Please add a systematic input-size study (e.g., 1, 2, 3, 4, or 5 input shells) with the same training and evaluation protocol, and ideally quantify the conditional variance of the true closure given the input state to justify the deterministic MLP mapping.","section":"Section III, Eqs. (2)-(3)"},{"comment":"The conclusion that the optimal time in the loop is around msteps = 100, or about 0.41 of the cutoff-shell eddy turnover time, is derived post hoc from a small set of msteps values, with no error bars and no repeated training runs for each msteps. The ratio 0.41 tau_Nc is therefore a fitted constant, not a parameter-free prediction, and the phrase 'a priori we expect' in Section IV belies the fact that the value is read off from the MSE curves. Furthermore, the study uses a single cutoff shell, a single Reynolds number, and a single forcing protocol, so the proposed physical rule is not demonstrated to generalize. Please provide uncertainty estimates for the optimal msteps, test at least one additional cutoff or Reynolds number, and present the time-in-loop comparison as a post hoc observation rather than an a priori derived law.","section":"Section IV, Figs. 4(b), 7(b), 10(b)"}],"minor_comments":[{"comment":"The displayed normalization in Eq. (4) is garbled and does not clearly show the denominator; please reformat the equation so the reader can see exactly how the MSE is normalized.","section":"Section III, Eq. (4)"},{"comment":"The caption says 'whereas θ denotes an explicit one' using the same symbol as the implicit case; this appears to be a typo and should use distinct notations for the explicit and implicit NN-dependent terms.","section":"Figure 2 caption"},{"comment":"The pdf of the eddy turnover time tau_n is shown without specifying the units of the horizontal axis; please state whether tau_n is expressed in ground-truth time steps, LES time steps, or absolute time.","section":"Section IV, Fig. 5"},{"comment":"The text contains a typo: 'wavenuber' should be 'wavenumber'.","section":"Section III"},{"comment":"The comparison with the LSTM and phenomenological closures is based only on local slopes of the second-order structure function; the paper would be strengthened by stating explicitly that this is one metric among several and by clarifying that the other comparisons are against the ground truth only.","section":"Section IV, Fig. 10"}],"recommendation":"major_revision","confidential_remarks":"The paper is a well-executed empirical study of solver-in-the-loop closures for a shell model, and the statistical validation is convincing for the single configuration considered. The main weaknesses — the unsupported locality assumption and the post hoc optimal-time rule — are fixable with targeted experiments, so rejection is not warranted. The recommended revision should focus on demonstrating the input-size convergence and on tempering the time-in-loop claim to reflect its fitted nature and limited scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi,\n\nThe paper is worth a serious look. The core result is that a solver-in-the-loop (unrolled) MLP closure for the Sabra shell model reproduces high-order intermittency statistics — flatness up to order 10, anomalous scaling exponents, Lagrangian structure functions, and the flux pdf including backscatter — at effectively Re ~ 1e12. That is a real step forward for data-driven closures, mainly because it shows a simple MLP trained with a differentiable solver can beat the static a priori approach and hold its own against an LSTM and a phenomenological model. The comparison set is thoughtful, and the error bars on the main statistics are computed in a sensible chunked way.\n\nWhat's genuinely new is the systematic study of the unroll horizon (msteps) as a hyperparameter and the attempt to tie the optimum to the eddy turnover time of the cutoff shell. That goes beyond Um et al.'s original solver-in-the-loop paper, which did not examine this.\n\nThe soft spots are real but not fatal. The locality assumption — that three pre-cutoff shells are enough input for the NN — is asserted with a single sentence and no quantitative convergence study. For Sabra, where interactions are only nearest and second-nearest, it's plausible, but the authors don't demonstrate that adding more shells or memory doesn't change the answer. The stress-test note worries this is load-bearing; I think it's a fair worry but the empirical agreement with GT is evidence that for this cutoff and dataset it works. Still, they should show the shell-count or memory ablation.\n\nThe 0.41 tau_Nc rule is the weakest part. It is derived after the fact from one configuration (Nc=14, Re=1e12), the MSE curves in Fig. 4 and 7 have no error bars, and the supporting pdf is just a histogram. The claim that the optimal time-in-loop is a fraction of the fastest-shell eddy turnover time is physically reasonable, but as presented it's a fitted constant, not a prediction. They need another cutoff or Reynolds number to test it.\n\nNo code or data is provided, which hurts reproducibility in a paper that is fundamentally about a training procedure. That should be fixed before publication.\n\nWho should read this: anyone working on differentiable physics or learned closures, and the shell-model community. It's a good paper with a clear empirical message. My advice: send it to review, but require code, an ablation on the input shells, and at least one more configuration to support the time-in-loop claim.","headline":"Solid demonstration that unrolled training beats a priori for shell-model closure, but the physical 'ideal time' claim is post hoc and the locality assumption is under-tested.","tokens_in":14990,"tokens_out":2451,"would_cite":true,"duration_ms":24764,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A neural closure trained inside the solver reproduces the highest-order statistics of shell-model turbulence.","keywords":["shell models","turbulence closure","solver-in-the-loop","differentiable physics","subgrid-scale modeling","intermittency","anomalous scaling","machine learning for fluids"],"falsifier":"A systematic convergence study on the number of input shells: if accuracy on the flatness of order 10 or on the flux pdf at the cutoff improves markedly when the input window is widened from three to, e.g., six shells, the locality assumption underlying the closure would be false. Alternatively, training the same method at a different cutoff (e.g., $N_c=20$) and checking whether the optimal unrolling time still scales with the new cutoff's eddy turnover time would test the paper's physical rule for $m_{\\text{steps}}$.","tokens_in":1671,"feed_emoji":"🌀","tokens_out":1721,"duration_ms":82003,"temperature":0.7,"pith_summary":"This paper argues that a subgrid-scale closure for a shell model of turbulence can be learned by training a neural network inside the differential equation solver itself, rather than on precomputed instantaneous input-output pairs. The central claim is that this solver-in-the-loop training produces a closure that reproduces the hardest statistical signatures of high-Reynolds-number turbulence: flatness up to order 10, the anomalous scaling of Eulerian structure functions, Lagrangian structure functions, and the full probability distribution of the energy flux at the cutoff, including the rare negative-flux backscatter events. The paper further claims that the optimal unrolling time, the number of solver steps the network sees during training, is set by the physics: about $0.41$ times the eddy turnover time of the fastest shell entering the loss. If these claims hold, a simple MLP closure trained a posteriori can outperform both a memory-based LSTM and a phenomenological closure, and the training horizon can be chosen from a physical timescale instead of tuned by hand.","feed_headline":"Solver-in-the-loop closure reproduces high-order turbulence statistics","feed_subtitle":"A simple MLP trained for a fraction of the cutoff eddy time matches intermittency and backscatter.","key_machinery":"The load-bearing mechanism is the differentiable solver. The reduced Sabra system is written in an automatic-differentiation framework; at each step the MLP outputs the two missing shells $\\tilde{u}_{N_c+1}, \\tilde{u}_{N_c+2}$ from the three preceding shells, and the Runge-Kutta integrator advances the resolved shells for $m_{\\text{steps}}$ times. Backpropagation then flows through the solver to update the MLP. The loss is the mean squared error between the LES-NN and ground-truth velocity fields over the six shells below the cutoff, so the network is trained under equation-consistent input distributions rather than static pairs.","core_discovery":"The authors demonstrate that a small MLP, trained for about $50$--$100$ solver steps $(\\approx 0.41 \\langle \\tau_{N_c}\\rangle)$, reproduces the fourth to tenth order flatness of the resolved shells, the anomalous scaling exponents $\\xi_p$, the Lagrangian structure functions up to order five, and the pdf of the convective flux at the cutoff, correctly capturing backscatter. The trained model remains stable when deployed for times far longer than the training window, and it keeps good accuracy even when the LES time step is increased to $10^{-4}$, i.e., $10{,}000$ times the ground-truth step. Comparisons against an LSTM-based closure and an optimal phenomenological closure show the solver-in-the-loop MLP is most accurate near the cutoff, especially on the local slopes of Eulerian structure functions. The authors interpret the unrolling time as physically meaningful: it should be a fraction of the eddy turnover time of the fastest shell in the loss, because beyond that the ground truth and the model decorrelate and the velocity-based loss becomes uninformative.","pith_inferences":["If the three-shell locality assumption survives a systematic convergence test, the computational cost of this closure is essentially independent of Reynolds number, which is not true for 3D LES and would be a decisive practical advantage.","The physical criterion for the optimal unrolling time may transfer to other chaotic multiscale systems where a differentiable solver is available, giving a first-principles rule for the training horizon instead of trial and error.","The authors speculate that a flux-based loss would not change the optimal unrolling time; testing that directly would clarify whether the velocity-based loss or the gradient dynamics is the limiting factor.","The stability of the model beyond its training window suggests the MLP has learned an approximation to the true subgrid transfer operator rather than a one-step correction, raising the possibility of transferring the trained closure to different forcings or cutoffs without retraining."],"forward_implications":["A simple MLP trained in the loop can reproduce intermittency and rare backscatter events that phenomenological closures miss, without needing a memory-based architecture.","The unrolling time can be set a priori from the eddy turnover time of the fastest shell in the loss (about $0.41 \\langle \\tau_{N_c}\\rangle$), removing a hyperparameter that is otherwise chosen arbitrarily.","The trained closure remains stable and accurate when deployed many eddy turnover times past its training window and at LES time steps up to $10^{-4}$ ($10{,}000\\times$ the ground-truth step).","The solver-in-the-loop MLP outperforms both an LSTM and an optimal phenomenological closure on local slopes of structure functions near the cutoff, the most sensitive region for closure error.","The same differentiable-physics training principle is put forward as a path toward subgrid closures for Navier-Stokes turbulence, where extreme subgrid fluctuations are far harder to capture."],"supporting_citations":[{"why":"Defines the Sabra shell model that the paper closes.","marker":"[16]"},{"why":"Provides the optimal phenomenological subgrid closure used as a baseline.","marker":"[30]"},{"why":"Provides the LSTM-based deep learning closure that the paper compares against.","marker":"[31]"},{"why":"Introduces the solver-in-the-loop training paradigm on which the method is built.","marker":"[33]"},{"why":"Studies the benefits of unrolling in time, supporting the analysis of the optimal time-in-the-loop.","marker":"[34]"}],"fun_headline_variants":["Tiny MLP trained briefly reproduces high-order turbulence statistics","Loop-trained MLP matches high-order turbulence stats","Solver-in-the-loop MLP stays stable long after training","Backscatter and high-order moments from a 50-step MLP","Loop-trained closure captures intermittency and backscatter"],"cache_read_input_tokens":17024,"weakest_assumption_plain":"The closure assumes that the velocity signals in the three shells just below the cutoff carry enough information to predict the subgrid energy transfer; if the true closure depends on a wider range of scales, the network's input window is insufficient.","fun_headline_variants_meta":{"raw":{"variants":["Tiny MLP trained briefly reproduces high-order turbulence statistics","Loop-trained MLP matches high-order turbulence stats","Solver-in-the-loop MLP stays stable long after training","Backscatter and high-order moments from a 50-step MLP","Loop-trained closure captures intermittency and backscatter"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000787,"raw_usage":{"total_tokens":3465,"prompt_tokens":934,"completion_tokens":2531,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":550,"completion_tokens_details":{"reasoning_tokens":2448}},"tokens_in":550,"tokens_out":2531,"duration_ms":18262,"temperature":1.0,"reasoning_tokens":2448,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:42:55.024204+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A systematic convergence study on the number of input shells: if accuracy on the flatness of order 10 or on the flux pdf at the cutoff improves markedly when the input window is widened from three to, e.g., six shells, the locality assumption underlying the closure would be false. Alternatively, training the same method at a different cutoff (e.g., $N_c=20$) and checking whether the optimal unrolling time still scales with the new cutoff's eddy turnover time would test the paper's physical rule for $m_{\\text{steps}}$.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Sabra shell model that the paper closes."},{"cited_title":"Biferale, A","cited_arxiv_id":null,"evidence_quote":"Provides the optimal phenomenological subgrid closure used as a baseline."},{"cited_title":"Ortali, A","cited_arxiv_id":null,"evidence_quote":"Provides the LSTM-based deep learning closure that the paper compares against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the solver-in-the-loop training paradigm on which the method is built."}],"review_version":1}