{"id":"dfadcdb7-7fa2-4d93-8faf-af444e8d614c","arxiv_id":"2412.13074","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Predicting the temporal derivative and integrating it with an ODE solver improves accuracy and stability of neural PDE surrogates compared with direct next-state prediction.","lead":"This paper tests training neural PDE surrogates to predict the rate of change of a solution and then using an ODE integrator to step forward in time, instead of directly predicting the next state. Across several PDEs and two model families, this change improves rollout accuracy and stability, and it allows flexible time stepping at inference.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The general applicability claim rests on finite-difference derivative labels being accurate, but the paper never measures label error against true derivatives; coarse-snapshot datasets could bias training labels and undermine the reported gains.","rationale":"The reader's weakest assumption is exactly the one I find most load-bearing: the derivative labels imported into Eq. (2) are assumed accurate enough to serve as ground truth. This assumption is not merely a detail; it is the bridge between 'saved solver output' and 'training target' for the entire derivative-prediction framework. If this bridge is biased, every downstream comparison is affected, including the headline accuracy gains and the claim that training on full-resolution data is beneficial. The paper addresses the computation of labels in principle (Richardson extrapolation) but never validates them against known or high-accuracy derivatives. Table 4's derivative error is relative to the same labels, so it cannot expose bias. The observed improvement from 2x-finer training data in Table 3 is consistent with label accuracy mattering, but it does not bound the bias at the native resolution actually used in the main experiments. I do not think this warrants rejection: the core empirical comparison is broad, code and data are released, and the paper is transparent about limitations and exceptions. However, the concern is real and unresolved, so the conditional verdict is appropriate. A direct label-accuracy experiment would either remove the concern or show that the method is sensitive to an unstated data-resolution requirement, which would narrow the claimed applicability.","tokens_in":22032,"tokens_out":7397,"duration_ms":82158,"concrete_test":"For 1D Advection, where analytic solutions are available, compute exact du/dt labels and train Unet/FNO derivative-prediction models with exact labels, with 4th-order Richardson labels at native dt, and with Richardson labels at 2x/4x coarser dt. If rollout error with exact labels is materially lower than with finite-difference labels at coarse dt, label bias is the limiting factor; if errors match at the resolutions used in Table 1, the concern is resolved. For NS/KS, generate reference trajectories at 10x finer temporal resolution and quantify the error of native-resolution Richardson labels against the fine-reference derivatives.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that the training labels y = du/dt in Eq. (2) faithfully represent the true temporal derivative. Section 3.1 computes these labels by finite differences and Richardson extrapolation from saved solver trajectories, but no experiment quantifies the error of these labels. Table 4 reports 'Derivative Error' as L(du/dt, F_theta(u)), i.e., against the same finite-difference labels, so it cannot detect label bias. In practical datasets, snapshots are often stored at coarse temporal resolution, and for chaotic systems (KS, NS, Kolmogorov flow) coarse central differences can be poor estimates of instantaneous derivatives. The paper's own Figure 6 shows small state noise is amplified in derivatives, and Table 3 shows that training on 2x-finer data (hence more accurate labels) improves NS results, confirming sensitivity to label resolution. Thus the broad-applicability claim is only as strong as label accuracy, which remains unmeasured.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an alternative training/inference framework for neural PDE surrogates: instead of predicting the next state u(t_{n+1}) from u(t_n), the network is trained to predict the temporal derivative ∂u/∂t at t_n, and a numerical ODE integrator (Forward Euler, Adams-Bashforth, Heun, RK4) advances the solution in time. The framework is evaluated with FNO and Unet architectures on six time-dependent PDEs (Advection, Heat, Kuramoto-Sivashinsky, Burgers, Navier-Stokes, Kolmogorov flow). The main empirical claims are that derivative prediction improves rollout accuracy and stability compared to direct state prediction, allows flexible time-stepping and integration schemes during inference, and enables training on more finely discretized data. The paper also compares against several training modifications (larger models, pushforward/unrolled training, PDE-Refiner), analyzes error sources via next-step error and a numerical oracle, and reports computational costs against numerical solvers.","tokens_in":22213,"tokens_out":5458,"duration_ms":52094,"significance":"If the central claim holds, this is a simple, architecture-agnostic improvement that could be broadly adopted by the neural surrogate community: it requires only changing the training target and adding an ODE integrator at inference. The paper is unusually thorough in its empirical scope (six PDEs, two architectures, multiple integrators, training and inference modifiers), and it releases code and datasets, which is a strength. The distinction between model error and integrator error in Section 5.2 is a useful contribution. However, the general applicability claim rests on the accuracy of finite-difference derivative labels, which is never directly validated, and the closest existing alternative (residual prediction) is discussed but never benchmarked. These gaps limit the strength of the conclusions as currently stated.","major_comments":[{"comment":"The training labels y = du/dt are computed from the saved simulation trajectories using finite differences and Richardson extrapolation, but the paper never measures the error of these labels against true derivatives. The 'Derivative Error' in Table 4 is evaluated against the same finite-difference labels, so it is a training loss rather than a measure of label fidelity. Since the framework's broad applicability depends on the availability of accurate derivative labels, the authors should quantify label error directly (e.g., by comparing to analytic derivatives for Advection or to high-resolution solver outputs for chaotic systems) and show how performance degrades as the snapshot spacing coarsens. Without this, the reported gains may be specific to datasets whose stored temporal resolution happens to be fine enough, and the paper's own Table 3 (2x steps/data) and Figure 6 suggest that label quality is indeed a sensitive factor.","section":"Sec. 3.1, Eq. (2); Sec. 4.1, Table 4"},{"comment":"The paper identifies residual prediction as the closest alternative and as a special case of derivative prediction with a fixed time step and Forward Euler integration, but no residual prediction baseline is included in any experiment. Given that residual prediction is widely used (e.g., Pfaff et al. 2021, Sanchez-Gonzalez et al. 2020, Stachenfeld et al. 2022, cited in Section 2), the claim that derivative prediction is a broadly beneficial framework requires a direct comparison against this baseline. Such a comparison would isolate the contribution of the higher-order derivative labels and the ODE integrator from the already-known benefits of predicting differences rather than absolute states; without it, the improvement over state prediction in Table 1 could be reinterpreted as a confirmation of residual-learning effects rather than evidence for the proposed framework.","section":"Sec. 2 (Related Work); Sec. 4 (Results)"}],"minor_comments":[{"comment":"The formula for Heun's method is missing a closing parenthesis: it should read 'u(t_{n+1}) = u(t_n) + (Δt/2)(F_θ(u(t_n), t_n) + F_θ(\\tilde{u}(t_{n+1}), t_{n+1}))'.","section":"Sec. 3.2, Heun's method"},{"comment":"The text says that initial-condition parameters are sampled from A_j, ω_j, l_j, and φ_j, but ω_j does not appear in Eq. (5); this appears to be a typo for l_j.","section":"Sec. 3.4, Eq. (5)"},{"comment":"'CF L' should be 'CFL' (e.g., 'CFL > 1'), and the phrase 'the resolution at which CF L = 1' in Figure 4's caption should be corrected.","section":"Sec. 4.4 (and elsewhere)"},{"comment":"The caption contains the fragment 'at sampled a, bvalues'; this should read 'sampled at a, b values'.","section":"Fig. 5 caption"},{"comment":"The sentence 'this is the first work to broadly apply this framework' is somewhat overstated given the prior derivative-prediction works cited in Section 2 (e.g., Sanchez-Gonzalez et al. 2019, Zeng et al. 2024); consider softening 'first' to 'one of the first to broadly evaluate across architectures and PDEs'.","section":"Sec. 6 (Conclusion)"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical study with potentially broad impact, but the two major comments above are central to the paper's main claim. The label-error issue is the most serious: if derivative labels are biased on coarser snapshot data, the generality claim fails, and the current Table 4 cannot detect this. A direct residual-prediction baseline is important for positioning the contribution. I would encourage the editor to request these additions and would be willing to review a revised version. The self-citations in related work are not problematic; the 'first work' claim in the conclusion is a minor overstatement."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"If you work on neural PDE surrogates, this is worth a look. The paper does a broad, careful comparison—two architectures, six PDEs, several integrators—and the central result is consistent: predicting du/dt and integrating with an ODE solver beats direct next-state prediction, and the advantage grows in finely discretized regimes. They also show flexible step sizes and that training on twice-resolved data helps. That's a genuinely useful empirical result, not a new mathematical framework.\n\nWhat's new: the systematic benchmark and the error decomposition (model vs integrator error, next-step vs derivative error, loss landscapes). Shipping code and data on GitHub/HuggingFace helps; the numbers are reproducible in principle. The loss landscape and numerical oracle analyses are post hoc but reasonable and labeled as such.\n\nWhere it's soft: First, the novelty claim in the conclusion—'first work to broadly apply this framework'—is contradicted by their own related-work section, which cites residual prediction in weather models and RK2-with-derivative GNNs. They acknowledge the idea 'is not novel' earlier, so the overclaim is inconsistent, not fraudulent. Second, the stress test is right: they never validate the finite-difference labels against true derivatives. Table 4's 'derivative error' is measured against the same finite-difference labels, so it cannot detect label bias. Their own Table 3 and Figure 6 show sensitivity to label resolution and noise, so the broad-applicability claim is only as strong as label accuracy, which remains unmeasured. Third, residual prediction—the most natural baseline—is discussed but never benchmarked. Since residual prediction is a scaled forward Euler derivative, the comparison to state prediction alone doesn't isolate what the framework adds over that prior practice.\n\nNone of this sinks the central empirical claim. The improvements are consistent across all six PDEs and both architectures, and the limitations section is honest about where derivative prediction fails (steady-state, implicit integrators). The paper is a solid contribution to the neural surrogate subfield.\n\nRecommendation: send it to peer review. It deserves referee time, and the label-accuracy and residual-baseline gaps are fixable with additional experiments.","headline":"A systematic, well-run empirical comparison of derivative-vs-state prediction for neural PDE surrogates; the main claims mostly hold, but the paper overstates novelty and never measures the accuracy of the derivative labels it trains on.","tokens_in":22684,"tokens_out":2159,"would_cite":true,"duration_ms":19649,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training neural PDE surrogates to predict the temporal derivative instead of the next state, then advancing the solution with a standard ODE integrator, improves rollout accuracy and stability across architectures and equations.","keywords":["neural PDE surrogates","temporal derivative prediction","ODE integration","state prediction","Fourier Neural Operator","autoregressive rollout","time-dependent PDEs","scientific machine learning"],"falsifier":"Train a derivative-prediction surrogate on a PDE with an analytic solution (for example, linear advection with known $u$) using derivative labels computed from progressively coarser saved trajectories while holding the inference step size fixed. If rollout error does not systematically drop below an equivalent state-prediction model trained on the same data once label bias is large, then the method's gain is an artifact of label accuracy rather than of the derivative objective.","tokens_in":21853,"feed_emoji":"🧮","tokens_out":7551,"duration_ms":59037,"temperature":0.7,"pith_summary":"This paper argues that neural surrogates for time-dependent partial differential equations should not be trained to jump directly from one solution snapshot to the next. Instead, the network should learn the instantaneous rate of change $\\partial u/\\partial t$ at a given state, and a conventional ODE integrator — Euler, Adams-Bashforth, Heun, or Runge-Kutta — should carry the solution forward at inference. The authors show this simple swap lowers rollout error and improves stability on advection, heat, Kuramoto-Sivashinsky, Burgers, Navier-Stokes, and Kolmogorov-flow benchmarks with both FNO and Unet architectures. Because the model is no longer tied to the dataset's time step, it can be trained on full-resolution simulation output and queried at arbitrary step sizes without retraining. If the results hold, derivative prediction is a cheap, broadly applicable upgrade to the standard state-prediction recipe.","feed_headline":"Predicting change beats predicting states in neural PDE solvers","feed_subtitle":"Swapping the training target to du/dt plus an ODE integrator cuts rollout error and enables flexible time-stepping.","key_machinery":"The central machinery is the derivative-prediction training objective paired with an ODE-integrator inference loop. Training minimizes $L_\\theta(u(t_n), t_n, y) = \\|F_\\theta(u(t_n), t_n) - y\\|_2^2$ with $y = \\partial u/\\partial t|_{t_n}$, where the labels are estimated from the stored trajectory using high-order finite differences and one-sided Richardson extrapolation at the endpoints. At inference the same network feeds its predicted derivative into Forward Euler, Adams-Bashforth, Heun's, or fourth-order Runge-Kutta updates, which is what decouples the learned spatial dynamics from the temporal advancement. The paper also decomposes error by comparing against a numerical oracle that integrates ground-truth derivatives, isolating integrator error from model error.","core_discovery":"The central claim is that \"by simply changing the training target and introducing numerical integration during inference, neural surrogates can gain accuracy and stability in finely-discretized regimes.\" Rather than fitting $u(t_{n+1}) = F_\\theta(u(t_n))$, the model is trained to satisfy $\\frac{\\partial u}{\\partial t}\\big|_{t_n} = F_\\theta(u(t_n))$, with labels computed from saved trajectories by finite differences and Richardson extrapolation. At inference an ODE integrator produces $\\hat{u}(t_{n+1})$ from the predicted derivative. The paper reports that this framework matches or beats state prediction on every tested PDE and architecture, that higher-order integrators add accuracy and permit larger steps, that models trained on twice the temporal resolution of the data achieve lower rollout error without extra inference cost, and that the benefit comes from a better-conditioned loss surface and from separating the learned spatial update from the integrated temporal update.","pith_inferences":["Inference (editorial): The same decoupling should transfer to other learned dynamical models — for example weather emulators, video prediction, or learned simulators with ODE structure — because the mechanism (a smoother loss surface for changes than for states) is not specific to PDEs.","Inference (editorial): The numerical-oracle experiment implies that once model error shrinks, integrator error becomes the binding constraint; pushing accuracy further may require learned correctors, implicit or exponential integrators, or adaptive step-size selection based on predicted derivative magnitude, none of which the paper implements.","Inference (editorial): The loss-landscape and noised-trajectory observations suggest derivative-trained models should be more sensitive to input perturbations during inference than state-trained models, because noise is amplified when differencing states; a targeted robustness study could test this directly.","Inference (editorial): If label accuracy is the key variable, the method's value depends on the availability of finely saved trajectories; for datasets archived at coarse temporal resolution, re-simulating or upsampling before computing derivative labels may be necessary to realize the advertised gains."],"forward_implications":["Training on full-resolution PDE data becomes viable: the same trajectory supplies many more training pairs without pinning the model to a tiny prediction step, and the paper reports lower rollout error on Navier-Stokes when trained on twice the data.","Inference can use higher-order integrators or different step sizes without retraining; Adams-Bashforth costs the same as one forward pass per step because past derivative predictions are cached, while Heun and RK4 trade extra model evaluations for larger stable steps.","Derivative prediction composes with existing training modifiers: the paper shows pushforward/unrolled training can be applied on top of derivative prediction for modest further gains on complex PDEs, and gains are comparable to or better than scaling model parameters.","The framework is limited to explicit time-stepping and time-dependent problems: implicit integrators are not covered, and steady-state or boundary-value problems would fail because the initial derivative does not encode the terminal solution."],"supporting_citations":[{"why":"Supplies the method-of-lines view that motivates predicting $\\partial u/\\partial t$ and integrating in time.","marker":"[1]"},{"why":"Supplies the FNO architecture and Navier-Stokes data setup used in the main experiments.","marker":"[3]"},{"why":"Supplies the Unet surrogate architecture used as the second model family.","marker":"[13]"},{"why":"Supplies the refiner-method baseline compared against derivative prediction in the training-modifier experiments.","marker":"[19]"},{"why":"Supplies the pushforward/unrolled training baseline and the stability concerns derivative prediction addresses.","marker":"[20]"},{"why":"Supplies the coarsened-solver timing methodology used in the computational-cost comparisons.","marker":"[26]"},{"why":"Supplies the Kolmogorov-flow drag term and the hybrid-solver context for the timing baseline.","marker":"[48]"},{"why":"Supplies the label-estimation scheme used to compute temporal-derivative training targets.","marker":"[60]"}],"fun_headline_variants":["Predict derivatives, not states, for better neural PDE solvers","Neural PDE surrogates gain stability from du/dt training","Flexible time-stepping achieved by predicting change in PDEs","Simple target switch improves neural PDE surrogate accuracy","ODE integration plus derivative targets lifts neural PDE performance"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the temporal-derivative labels computed from the saved simulation trajectories are accurate enough to serve as training targets; if the stored time resolution is too coarse, the finite-difference and Richardson-extrapolation labels become biased, and the predicted derivatives inherit that bias, eroding the claimed accuracy advantage.","fun_headline_variants_meta":{"raw":{"variants":["Predict derivatives, not states, for better neural PDE solvers","Neural PDE surrogates gain stability from du/dt training","Flexible time-stepping achieved by predicting change in PDEs","Simple target switch improves neural PDE surrogate accuracy","ODE integration plus derivative targets lifts neural PDE performance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000268,"raw_usage":{"total_tokens":1608,"prompt_tokens":926,"completion_tokens":682,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":542,"completion_tokens_details":{"reasoning_tokens":603}},"tokens_in":542,"tokens_out":682,"duration_ms":6517,"temperature":1.0,"reasoning_tokens":603,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:27:02.759813+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a derivative-prediction surrogate on a PDE with an analytic solution (for example, linear advection with known $u$) using derivative labels computed from progressively coarser saved trajectories while holding the inference step size fixed. If rollout error does not systematically drop below an equivalent state-prediction model trained on the same data once label bias is large, then the method's gain is an artifact of label accuracy rather than of the derivative objective.","supporting_citations":[{"cited_title":"Elsevier (2012)","cited_arxiv_id":null,"evidence_quote":"Supplies the method-of-lines view that motivates predicting $\\partial u/\\partial t$ and integrating in time."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the label-estimation scheme used to compute temporal-derivative training targets."}],"review_version":1}