{"id":"2336b746-3b04-400c-8497-26ce0826e633","arxiv_id":"2505.11308","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Closure models for under-resolved PDEs can be trained on synthetic manufactured solutions, and the learned corrections generalize from forced to unforced equations.","lead":"This paper trains reinforcement learning agents to correct coarse-grid simulations of PDEs, using synthetic solutions generated by the method of manufactured solutions instead of expensive fine-grid data. The method is demonstrated on Burgers and advection equations, including extrapolation to unforced versions of the equations.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Homogeneous generalization is confounded: the policy is selected using a homogeneous validation set, so the reported 40–80% error reductions may reflect checkpoint selection rather than transfer from MMS.","rationale":"The reader's weakest_assumption focuses on MMS representativeness, but the paper's own Appendix A reveals a more direct threat to the central generalization claim: the model is selected using homogeneous validation data. This is a circular step, because the claim is that closures trained only on inhomogeneous MMS data transfer to homogeneous PDEs; using the homogeneous validation set to pick a checkpoint means the target distribution has influenced model selection. The reader did note this in the rationale, but did not make it the primary weak assumption. I therefore partially agree. The MMS representativeness concern is real as a scope limitation, especially because the training and test initial conditions are all smooth, low-wavenumber trigonometric/exponential functions; however, the checkpoint-selection issue alone can explain the reported homogeneous gains without any genuine transfer, making it more load-bearing. The in-distribution results are not affected by this concern, which is why I do not recommend rejection outright. The paper should be accepted only conditionally, with the out-of-distribution evaluation rerun under a model-selection protocol that does not use homogeneous data, and with the selected epoch and last-epoch performance reported.","tokens_in":12986,"tokens_out":4758,"duration_ms":51658,"concrete_test":"Run the 1D Burgers experiment twice with identical hyperparameters and training data. In run A, select the checkpoint using the current homogeneous validation protocol. In run B, select the checkpoint either at a fixed epoch, e.g., the last epoch, or using only a held-out set of inhomogeneous MMS cases. Evaluate both checkpoints on the same 30 homogeneous test trajectories and compare median cumulative MSE reduction. If run B's reduction is substantially below run A's or is not significantly positive, the claimed homogeneous generalization is an artifact of model selection rather than of MMS-based training.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Appendix A states that at the end of each epoch the RL model is evaluated on 32 homogeneous-PDE validation trajectories and that this validation set is used to choose the checkpoint that performs best on both homogeneous and inhomogeneous PDEs. The central out-of-distribution claim—that closures trained only on forced MMS data generalize to unforced homogeneous PDEs—therefore is not tested as stated: the reported homogeneous test errors come from a model whose selection already saw homogeneous data. Since the homogeneous task is exactly the extrapolation target, this is a form of target-distribution leakage. It is especially consequential in this setup because PPO training is noisy and different checkpoints can vary substantially; selecting the best of 1000 epochs on the homogeneous validation set can produce large apparent gains even if most trained policies do not transfer. The paper does not report the selected epoch, the validation scores, or the performance of the final or an inhomogeneous-validation-selected checkpoint, so the 80%, 60%, and 40% reductions cannot be attributed to the MMS training procedure. Fixing this is a prerequisite for the central claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a framework for learning closure models for coarse-grained PDE solvers using reinforcement learning trained exclusively on synthetic data generated by the method of manufactured solutions (MMS). Instead of running expensive fine-grid simulations, the authors manufacture parameterized analytical solutions, compute the corresponding forcing terms, and train an RL policy (Closure-RL) to correct the coarse-grid update. The method is demonstrated on 1D Burgers, 2D Burgers, and 2D advection equations, with in-distribution tests on unseen MMS-forced cases and out-of-distribution tests on homogeneous (unforced) PDEs. The central claim is that closures trained on inhomogeneous MMS data generalize to homogeneous PDEs, achieving median error reductions of roughly 80%, 60%, and 40% for the three test families relative to the unclosed coarse-grid solver. The paper also compares against a Fourier Neural Operator baseline and reports training details in the appendix.","tokens_in":13202,"tokens_out":4389,"duration_ms":43337,"significance":"If the central claim holds, the work is significant: it offers a way to train data-driven closure models without any fine-grid simulation data, which is a major bottleneck for many PDE applications. The strengths of the paper are that the synthetic-data pipeline is simple and clearly described, the empirical results are reported consistently across three PDE families with interquartile ranges, the training configuration is documented in detail, and the RL closures are compared with an FNO baseline. However, the significance is conditional: the homogeneous generalization claim is currently weakened by the checkpoint-selection procedure described in Appendix A, and the MMS solution families are smooth and low-wavenumber, so the scope of the demonstrated extrapolation needs to be stated more carefully. With the selection confound resolved, the paper would constitute a meaningful contribution to closure modeling and data-efficient learning for PDEs.","major_comments":[{"comment":"The homogeneous out-of-distribution claim is confounded by checkpoint selection. Appendix A states that at the end of each epoch the model is evaluated on a validation set consisting of 32 homogeneous-PDE initial conditions, and that this validation set is used to choose the checkpoint that performs best on both homogeneous and inhomogeneous PDEs. Since the homogeneous task is exactly the extrapolation target, the reported median error reductions (roughly 80%, 60%, and 40% in Figures 4, 7, and 9) may reflect selecting the best of 1000 PPO checkpoints on the target distribution rather than transfer from the MMS training procedure. The manuscript does not report the selected epoch, the validation scores, or the performance of the final checkpoint or of a checkpoint selected using only inhomogeneous validation data. Please provide these results; without them, the central claim that MMS-trained closures generalize to homogeneous PDEs is not tested as stated.","section":"Appendix A; §3.1.2, §3.2.2, §3.3.2"},{"comment":"The representativeness of the MMS solution families for the homogeneous extrapolation is not established. The manufactured solutions are smooth trigonometric and exponential functions with wavenumbers 2π or 4π and decaying amplitudes, and the homogeneous test cases appear to be initialized from the same MMS family (the text in §3.1.2 refers to \"30 different MMS solutions\" for a homogeneous problem, which has no MMS forcing). The paper therefore demonstrates transfer from forced to unforced dynamics within one smooth, low-wavenumber solution family, not transfer to homogeneous evolutions with qualitatively different features such as shocks or sharp gradients. A concrete test would be to evaluate the trained closure on homogeneous initial conditions that are not drawn from Eqs. (9), (11), or (14), for instance a localized pulse or a developing shock; if the closure degrades there, the abstract and conclusions should be narrowed accordingly.","section":"§2.1, Eqs. (9), (11), (14); §3.1.2"}],"minor_comments":[{"comment":"The homogeneous test is described as being evaluated \"across 30 different MMS solutions,\" but a homogeneous problem has no MMS forcing term; please state explicitly that the 30 test initial conditions are drawn from Eq. (9) at t=0 (or otherwise specify the initialization procedure).","section":"§3.1.2 and Figure 4"},{"comment":"The caption of Figure 18 says \"Results for the inhomogeneous 1D Burgers' equation,\" but the surrounding text in Appendix D describes this figure as the homogeneous, out-of-distribution test case; please correct the caption.","section":"Figure 18 caption"},{"comment":"The word \"tubulence\" in the first paragraph of the Introduction should be \"turbulence.\"","section":"Page 2, Section 1"},{"comment":"The notation fΔx = dΔx is confusing because it reads as multiplication by f; please clarify that the fine-grid spacing is related to the coarse-grid spacing by a scaling factor d (for example, Δx_f = d Δx).","section":"Section 2.2"},{"comment":"Please state explicitly in the main text that during training the target S(ψ^n) in the reward is the analytical MMS solution, not a numerically simulated fine-grid solution, since the relevant remark currently appears only in Section 2.2.","section":"Section 2.3, Eq. (7)"}],"recommendation":"major_revision","confidential_remarks":"The main issue is the checkpoint-selection leakage: because the model is selected on a homogeneous validation set, the reported out-of-distribution improvements may not demonstrate transfer from MMS training. If the authors can show that a final checkpoint or a checkpoint selected on inhomogeneous validation data still reduces homogeneous test errors substantially, the central claim would be much stronger. The FNO comparison in Appendix D learns a full solution operator rather than a closure, so it is not a direct baseline for closure modeling; the comparison is still informative but should be framed accordingly. The reliance on the authors' own Closure-RL framework is appropriate given that the paper explicitly builds on it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: the paper has a genuine new idea — training RL closures for coarse-grid PDEs on synthetic data from the method of manufactured solutions, avoiding the need for expensive fine-grid training data. The in-distribution results on 1D Burgers, 2D Burgers, and 2D advection are credible: median error reductions of 40–80% with reasonable interquartile ranges, and a fair comparison to an FNO baseline in the appendix. The paper is clearly written and the limitations section is honest about requiring full knowledge of the PDE.\n\nThe soft spot is exactly what the stress-test note flags. Appendix A states that at the end of every epoch the RL model is validated on 32 homogeneous-PDE trajectories, and that this validation set is used to choose the checkpoint that performs best on both homogeneous and inhomogeneous PDEs. That means the reported 80%/60%/40% error reductions for the homogeneous case come from a model selected on the very distribution that is supposed to be the extrapolation target. This is target-distribution leakage. RL training is noisy — different checkpoints vary a lot — and picking the best of 1000 epochs on the homogeneous validation set can produce large apparent gains even if most trained policies do not transfer. The paper does not report the selected epoch, the validation scores, or the performance of the final checkpoint, so the out-of-distribution claim is not cleanly established.\n\nA second, lesser issue: there are no comparisons to standard closure baselines (e.g., Smagorinsky-type or simple data-driven closures) — only the FNO comparison, which is not a closure model. That limits the interpretation of the in-distribution gains.\n\nA minor but real limitation: the manufactured solution families are smooth, low-wavenumber trigonometric/exponential functions. The paper gives no evidence that this family captures the subgrid-scale behavior outside that smooth regime, so the extrapolation to homogeneous PDEs — even setting aside the checkpoint selection issue — remains a claim about a fairly narrow solution class.\n\nWho is it for: people working on data-driven closures and surrogate PDE models. It deserves serious review, but with a request for re-analysis or explicit reporting of the checkpoint selection. The easiest fix is to either select checkpoints without homogeneous validation data, or report results for the final policy and for a policy selected on inhomogeneous validation only. If the homogeneous transfer survives that, it is a solid contribution; if not, the in-distribution story still stands.\n\nRecommendation: send to peer review with major revision — the central claim needs the selection confound addressed before publication.","headline":"Real idea with solid in-distribution gains, but the headline out-of-distribution claim is weakened because the model checkpoint was selected on homogeneous validation data.","tokens_in":13693,"tokens_out":1494,"would_cite":true,"duration_ms":16593,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["68T05","35Q53","65M06"],"pacs":[],"model":"deepseek-v4-flash","headline":"A reinforcement-learning policy can learn a closure model for a coarse-grid PDE from synthetic manufactured solutions alone, and the same policy cuts errors on the unforced PDE it never saw.","keywords":["reinforcement learning","closure models","method of manufactured solutions","coarse-grained PDEs","Burgers' equation","advection equation","synthetic training data","subgrid-scale modeling"],"falsifier":"Run the trained one-dimensional Burgers closure on homogeneous initial data containing a sharp front or high-wavenumber content not present in the training family, and compare its cumulative squared error against the unclosed coarse solver; if the closure does not beat the coarse solver across the simulation, the transfer claim fails.","tokens_in":12789,"feed_emoji":"🧮","tokens_out":7152,"duration_ms":67358,"temperature":0.7,"pith_summary":"This paper claims that a reinforcement-learning closure model for an underresolved PDE can be trained entirely on synthetic data generated by the method of manufactured solutions, with no fine-grid simulation performed during training. The trained closure acts as an additive correction at each coarse-grid time step. On unseen inhomogeneous cases it cuts the median squared error of the coarse solver by roughly 40 to 80 percent, and the same policy also improves the unforced homogeneous PDE, which it never saw in training. If this transfers beyond the smooth, low-wavenumber test families, it would make data-driven closure modeling practical for systems where high-resolution training data are scarce.","feed_headline":"Synthetic data alone can train PDE closures that cut error by 80%","feed_subtitle":"Reinforcement learning corrections transfer to unforced equations never seen in training.","key_machinery":"The central objects are the method of manufactured solutions and the Closure-RL policy. The method of manufactured solutions turns a chosen smooth analytic function into an exact PDE solution by adding a computed forcing term to the equation. The Closure-RL policy is a convolutional neural network trained with proximal policy optimization that observes the normalized forcing term, the last two coarse states, and the PDE parameters, and outputs a corrective action added to the coarse-grid update. The reward at each step is the reduction in squared error between the coarse state and the subsampled manufactured solution, and the forcing term in the observation is what lets the policy transfer from inhomogeneous training cases to homogeneous evolutions where the forcing is zero.","core_discovery":"The central claim is that the missing subgrid-scale information in a coarse-grid PDE solve can be learned as a per-point forcing correction by a reinforcement-learning policy, with all training data obtained by manufacturing smooth analytic solutions and back-computing their forcing terms. Because the manufactured solutions are known analytically, the fine-grid reference used for rewards is available without any expensive simulation. The paper shows on the one- and two-dimensional Burgers equations and the two-dimensional advection equation that the learned closure reduces median squared error by more than 80 percent, 60 percent, and 40 to 80 percent, respectively, on unseen inhomogeneous cases, and reduces it by roughly 80 percent, 40 to 60 percent, and 40 percent on homogeneous evolutions. In a direct comparison, the reinforcement-learning closure also generalizes to these out-of-distribution homogeneous cases better than a Fourier neural operator trained on the same synthetic data.","pith_inferences":["This suggests the generalization hinges on the forcing term in the observation: at test time the policy sees a zero forcing signal and must rely on the coarse state alone, which may explain why out-of-distribution gains are smaller than in-distribution gains.","A natural stress test is to train on smooth low-wavenumber families and evaluate on shock-forming or turbulent initial data; the paper's claim would fail precisely where the manufactured family stops being representative.","Because the method of manufactured solutions supplies exact pointwise targets, the same synthetic data could train non-reinforcement-learning closure models through differentiable solvers, widening the approach beyond policy-gradient training.","The low-wavenumber exponential-decay structure of all three manufactured families suggests the method is most credible for regimes where the unresolved scales are smooth; evidence for broadband or chaotic subgrid dynamics is still open."],"forward_implications":["Closure models can be trained without computing any fine-grid reference simulations, because the manufactured solution provides the reference analytically.","A closure trained on forced, inhomogeneous PDEs transfers to unforced homogeneous evolutions, so the training distribution does not have to contain the target PDE regime.","Across the three test PDEs, the learned closures reduce the median squared error of the coarse solver by roughly 40 to 80 percent on unseen cases.","The same synthetic-data pipeline can train other models such as neural operators, but the reinforcement-learning closure generalizes better to out-of-distribution unforced cases.","Systems whose governing equations are known but whose high-resolution data are scarce become viable targets for data-driven closure modeling."],"supporting_citations":[{"why":"Supplies the method of manufactured solutions used to generate all synthetic training data.","marker":"[Roache, 2002]"},{"why":"Supplies the Closure-RL framework and its grid-based reinforcement-learning architecture, which this paper adapts for synthetic data.","marker":"[von Bassewitz et al., 2025]"},{"why":"Supplies proximal policy optimization, the algorithm used to train the closure policy.","marker":"[Schulman et al., 2017]"},{"why":"Defines the Fourier neural operator baseline used for the out-of-distribution comparison in the appendix.","marker":"[Li et al., 2020]"},{"why":"Supplies the implementation of the Fourier neural operator baseline used in the comparison.","marker":"[Kossaifi et al., 2024]"}],"fun_headline_variants":["RL closures trained on synthetic data cut PDE error by 80%","Synthetic-data RL closures slash coarse-grid PDE error 80%","Reinforcement-learned PDE closures generalize from synthetic data","Synthetic data trains RL closures that cut PDE error 80%","RL closure models from synthetic data beat coarse-grid PDEs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the parameterized, smooth, low-wavenumber manufactured solutions used for training represent the true solutions of these PDEs closely enough that a closure learned on them will also correct coarse-grid errors for unforced evolutions.","fun_headline_variants_meta":{"raw":{"variants":["RL closures trained on synthetic data cut PDE error by 80%","Synthetic-data RL closures slash coarse-grid PDE error 80%","Reinforcement-learned PDE closures generalize from synthetic data","Synthetic data trains RL closures that cut PDE error 80%","RL closure models from synthetic data beat coarse-grid PDEs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.0006,"raw_usage":{"total_tokens":2784,"prompt_tokens":903,"completion_tokens":1881,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":519,"completion_tokens_details":{"reasoning_tokens":1794}},"tokens_in":519,"tokens_out":1881,"duration_ms":13442,"temperature":1.0,"reasoning_tokens":1794,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:53:21.654138+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the trained one-dimensional Burgers closure on homogeneous initial data containing a sharp front or high-wavenumber content not present in the training family, and compare its cumulative squared error against the unclosed coarse solver; if the closure does not beat the coarse solver across the simulation, the transfer claim fails.","supporting_citations":[{"cited_title":"Code verification by the method of manufactured solutions","cited_arxiv_id":null,"evidence_quote":"Supplies the method of manufactured solutions used to generate all synthetic training data."},{"cited_title":"Closure discovery for coarse-grained partial differential equations using grid-based reinforcement learning","cited_arxiv_id":null,"evidence_quote":"Supplies the Closure-RL framework and its grid-based reinforcement-learning architecture, which this paper adapts for synthetic data."}],"review_version":1}