{"id":"e1f3648c-96de-4451-8ecf-66d5c398bafd","arxiv_id":"2505.10930","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A self-supervised framework that discovers governing equations from short observed data windows and uses them to regularize autoregressive PDE foundation models, improving long-term forecast accuracy.","lead":"This paper introduces a training add-on for AI models that predict physical systems, discovering the hidden equations from the system's own data and using those equations to keep predictions physically consistent. It reports lower long-term forecast error on several physics benchmarks, which matters for any application that relies on AI simulation of fluids or waves.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Consistency loss has zero gradient under the stated ADO, so the temporal-alignment mechanism cannot produce the reported ablation (Eqs. 8-11).","rationale":"The reader's weakest_assumption flags coefficient comparability and normalization, which is a real but secondary issue. The more load-bearing problem is that the consistency loss, as specified, cannot update the network parameters at all. This directly undermines the abstract's core claim ('aligns the physical dynamics discovered at different time steps') and makes the ablation in Table 3 Task 1 internally inconsistent. I do not see this as an accusation of misconduct; it is an incomplete specification of the optimization that must be resolved by the code or a revised description. The empirical comparisons in Table 1 might still be valid if the code uses a differentiable surrogate, so the appropriate verdict remains conditional rather than reject. I recommend keeping the reader's CONDITIONAL verdict but adding the explicit requirement to demonstrate nonzero gradient and disclose the surrogate. If the code confirms zero gradient and the ablation cannot be reproduced, the central mechanism collapses and the paper should be rejected; the one concrete test above settles this.","tokens_in":43433,"tokens_out":12205,"duration_ms":129952,"concrete_test":"Inspect the released code at https://github.com/SCAILab-USTC/PITA and identify the exact theta-update used for LCon. Then compute grad_theta LCon at initialization (and after one ADO iteration) using the code's Algorithm 1 implementation. If the gradient norm is identically zero because Lambda is frozen, re-run the Task 1 LData+LCon ablation on FNO-NS-1e-3 with the DPOT-Ti setting; if the result does not reproduce Table 3's 0.00165/0.00249, the published numbers cannot come from Eq. (8) as described. If the code instead unrolls or relaxes the thresholding/solver to make Lambda differentiable, verify that the resulting surrogate is what Eq. (8) denotes and measure how much its coefficients deviate from Algorithm 1's output; this deviation is an undocumented part of the method that must be disclosed for the central claim to be reproducible.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper's novelty is the consistency loss LCon in Eq. (8), defined as sum over rollout windows of ||Lambda* - Lambda_i(theta)||^2, with Lambda_i(theta) the sparse-regression coefficients on predicted frames. But the alternating-direction procedure in Sec. 3.5 makes LCon unable to influence theta. Eq. (10) computes Lambda^{k+1} from the frozen theta^k, and Eq. (11) then updates theta with that Lambda^{k+1} held fixed. Since LCon depends on theta only through Lambda_i(theta), it is constant during the theta update, so grad_theta LCon = 0. The only way to get a nonzero gradient is to treat Lambda_i as a differentiable function of theta through Algorithm 1, but Algorithm 1 uses hard thresholding and a pseudoinverse, and no differentiation rule or unrolling/relaxation is given. This is not a minor implementation detail: Table 3 Task 1 reports that LData+LCon reduces nRMSE from 0.00301/0.00437 to 0.00165/0.00249, which is impossible if the LCon term has zero gradient. Either the released code uses an undocumented surrogate (e.g., straight-through estimator, soft thresholding, implicit differentiation), or the ablation/algorithm description is inconsistent. If LCon is in fact inert, the claimed temporal-alignment mechanism is not what drives the reported 30% long-trajectory and 37.8% Burgers improvements; the gains would need to be attributed to LPhy (Eq. 7) alone, which is a known PINN-style regularization rather than the paper's new alignment idea. Related but secondary: even if a gradient path existed, comparing Lambda* (full-resolution 10 frames) with Lambda_i (quarter-resolution, TC=3 frames) requires column normalization across windows, which is unspecified.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes PITA, a self-supervised framework that adds physics-informed temporal alignment to auto-regressive PDE foundation models. The method first discovers a sparse coefficient vector Λ* from the initial ground-truth frames of a trajectory via sparse regression, then, during training, discovers coefficient vectors Λ_i(θ) from predicted rollout windows and penalizes their deviation from Λ* with a consistency loss; a physics residual loss and uncertainty-based weighting are added, and an alternating direction scheme is used. Experiments on FNO, PDEBench, PDEArena, and CFDBench across DPOT, MPP, and FNO backbones report consistent nRMSE reductions, including roughly 30% average improvement on long trajectories and 37.8% on an out-of-distribution Burgers task.","tokens_in":43786,"tokens_out":5387,"duration_ms":54093,"significance":"If the mechanism works as claimed, PITA is a practically valuable plug-in for improving long-horizon accuracy and OOD robustness of autoregressive PDE foundation models, requiring no known PDE form. The paper's strengths are its broad empirical coverage (multiple foundation models, model sizes, and datasets), component-wise ablations, robustness and noise experiments, and a computational-cost analysis. The central claim, however, rests on the consistency loss being able to influence the network parameters, and the manuscript does not specify how gradients flow through the sparse-regression step; it also does not justify the comparability of coefficient vectors obtained at different resolutions and sampling densities. These are load-bearing gaps rather than presentation issues.","major_comments":[{"comment":"As written, the alternating-direction procedure makes LCon unable to influence θ. In Eq. (10), Λ^{k+1} is computed from the frozen θ^k, and in Eq. (11) θ is updated with that Λ^{k+1} held fixed. Since the only θ-dependence of LCon in Eq. (8) enters through Λ_i(θ), the consistency term is constant during the θ update unless Λ_i(θ) is differentiated through Algorithm 1. Algorithm 1 uses hard thresholding and a pseudoinverse, with no stated differentiation rule, unrolling, or relaxation. Table 3, Task 1, reports that LData+LCon reduces nRMSE from 0.00301/0.00437 to 0.00165/0.00249, which is impossible if the term has zero gradient. Please specify the actual gradient path (e.g., straight-through estimator, soft-thresholded surrogate, implicit differentiation, or unrolled sparse regression) and update the algorithm description accordingly.","section":"Sec. 3.5, Eqs. (10)-(11)"},{"comment":"The consistency loss compares Λ* obtained from the first ten full-resolution ground-truth frames with Λ_i(θ) obtained from predicted windows that are downsampled to TC=3 temporal frames and one-quarter of the spatial points. Sparse-regression coefficients are not invariant to the finite-difference stencil spacing used for derivative approximation, nor to the column scaling of the library Φ(θ). The manuscript does not state any column normalization or resampling correction. If the columns of Φ have different norms in the ground-truth and predicted-window constructions, ∥Λ*−Λ_i(θ)∥² is a scale-sensitive quantity and may not measure physical disagreement. Please specify the normalization and justify comparability, or the consistency target is not well-defined.","section":"Sec. 3.3 and Sec. 3.4, Eq. (8)"},{"comment":"The physics loss LPhy is written with Λ_i, which appears to be the sparse-regression solution for the same predicted window from Eq. (4). In that case the residual ∥∂tU_i−Φ_iΛ_i∥² is already minimized by Algorithm 1 and is approximately zero, so its role in Eq. (11) is unclear. If Λ_i is instead a free variable in Eq. (7), the alternating procedure in Eqs. (10)-(11) does not state how it is updated in the θ-step. Please clarify the status of Λ_i in LPhy and how gradients flow through it; this is needed to interpret the Task 1 ablation attribution between LPhy and LCon.","section":"Sec. 3.4, Eq. (7)"}],"minor_comments":[{"comment":"The main results table reports no error bars or standard deviations, whereas Appendix E.12 provides them only for long-trajectory datasets; please state in the main text or table caption that variance information is deferred to the appendix, or add representative error bars.","section":"Table 1 and Appendix E.12"},{"comment":"The text says \"One-Order\" removes only second-order derivatives, but the table and surrounding discussion suggest that first-order derivative terms are present in that configuration; please clarify the exact library composition for each row of Task 5.","section":"Sec. 4.4, Task 5"},{"comment":"There are typos in the appendix captions: \"visulize\" (F.3), \"naive-stokes\" (F.4), and \"PDE-Bench CNS-(M,η)\" is inconsistent with the \"PDEBench-CNS\" naming used elsewhere; these should be corrected.","section":"Appendix F.3 and F.4"},{"comment":"The scalar α appears both in the sparse-regression objective Eq. (4) and in LPhy Eq. (7); please state explicitly whether these are the same hyperparameter and how α is set in the experiments.","section":"Eqs. (4) and (7)"},{"comment":"The diagonal downsampling matrix D is mentioned in the figure and in Sec. 3.3 but is not used in any subsequent equation; either remove it or define its role formally in the compressed-data notation.","section":"Fig. 2 and Sec. 3.3"}],"recommendation":"major_revision","confidential_remarks":"The empirical contribution is substantial and the reported gains are consistent, but the core temporal-alignment mechanism is not reproducible from the text because the gradient through the sparse-regression step is unspecified, and the coefficient comparability across differently downsampled windows is not established. I did not inspect the released code; if the code implements a differentiable surrogate for sparse regression, a short description of that surrogate would resolve the main issue. The 'first work' novelty claim in Sec. 3.4 should also be softened or scoped to the specific formulation used here."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things you should know about arXiv:2505.10930. First, the core mechanism—the consistency loss that aligns sparse-regression coefficients from predicted windows with ground-truth coefficients—is not trainable under the algorithm as written. The alternating procedure in Sec 3.5 fixes theta and solves for Lambda in Eq (10), then updates theta with that Lambda held fixed in Eq (11). Since LCon only depends on theta through Lambda_i(theta), its gradient with respect to theta is identically zero. The ablation in Table 3 showing LData+LCon cutting nRMSE from 0.00301 to 0.00165 is therefore impossible unless the released code uses an undocumented surrogate gradient (straight-through, soft threshold, or similar). The paper never mentions one.\n\nThat is a load-bearing problem, not a typo. If LCon is inert, the reported 30% long-trajectory and 37.8% Burgers improvements would have to come from the physics loss alone, which is a known PINN-style regularizer. The paper's novelty would collapse to a rediscovery of that term.\n\nWhat is genuinely good: the idea of extracting a PDE from the first frames and then checking discovered laws on later windows is a sensible way to inject self-supervised physics into autoregressive foundation models. The evaluation is wide: multiple foundation models, many PDE datasets, ablations of each loss component, and the code is public. The no-known-physics-prior framing is honest, and the experiments show large and fairly consistent improvements if the mechanism actually works.\n\nThe reader's secondary concern is also valid. Lambda* comes from full-resolution ten ground-truth frames, while Lambda_i comes from quarter-resolution windows of three frames. Comparing those coefficient vectors directly requires column normalization that isn't specified. Even with a proper gradient path, the fairness of that comparison is unclear. Also, the main tables lack error bars; the appendix adds some for long trajectories, but the headline numbers in Table 1 are single runs.\n\nBottom line: this is a promising direction with a serious gap in the description of the central loss. A specialized reader building physics-constrained fine-tuning for PDE foundation models would benefit from engaging with the code and the idea. I would not cite it as-is, but I would send it to an expert referee, asking them to verify the gradient path in the code and to demand the authors specify how Lambda is differentiated (or admit the consistency loss is not what drives the gains). The paper deserves a serious look, but as written the math says the main lever is inert.","headline":"The paper's central consistency loss has zero gradient under the stated alternating optimization, which undermines the reported ablation unless the code uses an undocumented surrogate; still worth a peer review round to pin down the mechanism.","tokens_in":44304,"tokens_out":3404,"would_cite":false,"duration_ms":34644,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"By aligning the governing equations discovered from ground-truth and predicted windows, PITA reduces long-trajectory prediction error of auto-regressive PDE foundation models by about 30%.","keywords":["physics-informed temporal alignment","auto-regressive PDE foundation models","shortcut learning","error accumulation","PDE discovery","sparse regression","self-supervised learning","out-of-distribution generalization"],"falsifier":"Finetune a PDE foundation model with PITA on a trajectory whose governing equation includes a term deliberately left out of the candidate library, for example a cubic damping term when the library only contains quadratic terms. If the consistency loss then forces the recovered coefficients toward a $\\Lambda^*$ that cannot represent the true law, and prediction error stays at or above the plain auto-regressive baseline, the reported gains rely on assuming the library contains the relevant terms rather than on the alignment mechanism itself.","tokens_in":43262,"feed_emoji":"🌊","tokens_out":7321,"duration_ms":67514,"temperature":0.7,"pith_summary":"Auto-regressive PDE foundation models tend to learn a shortcut: instead of evolving the field, they copy the previous frame, and errors pile up over long rollouts, especially on out-of-distribution data. This paper proposes a self-supervised training signal, PITA, that makes the model check its own output against the governing equation it can infer from the data. After each rollout, sparse regression on a library of candidate derivative and polynomial terms recovers a coefficient vector from the ground-truth window and from each predicted window; the model is penalized when these vectors disagree, and also when the predicted field fails to satisfy its own recovered equation. On twelve datasets and models up to 500 million parameters, the method reduces long-trajectory error by about 30% on average and improves out-of-distribution accuracy on a previously unseen Burgers dataset by 37.8%.","feed_headline":"PDE forecast error cut ~30% by aligning discovered physics","feed_subtitle":"A self-supervised loss re-checks the governing equation each rollout, improving long and out-of-distribution forecasts.","key_machinery":"The load-bearing object is the discovered coefficient vector $\\Lambda$ from sparse regression on a candidate library. Given compressed data $U(\\theta)$, the method builds a matrix $\\Phi(\\theta)$ whose columns are candidate terms—state values, spatial derivatives, low-degree polynomials, and a constant—and solves $\\partial_t U(\\theta) = \\Phi(\\theta) \\Lambda$ under $\\ell^0$ sparsity via thresholded ridge regression. PITA computes $\\Lambda^*$ from a full-resolution ground-truth window and $\\Lambda_i$ from each downsampled predicted window, then penalizes their difference; this temporal alignment is the new supervision mechanism. An alternating-direction optimizer alternates sparse-regression updates of $\\Lambda$ with gradient updates of the neural operator, and uncertainty-based weights balance data, physics, and consistency losses.","core_discovery":"The central claim is that the shortcut problem in auto-regressive PDE prediction can be suppressed without any known-physics prior by aligning the physics discovered at different times on each trajectory. Because any segment of a trajectory is governed by the same law, the coefficient vector $\\Lambda^*$ recovered by sparse regression from the first ten full-resolution ground-truth frames should equal the coefficient vector $\\Lambda_i$ recovered from each downsampled predicted window. PITA enforces this equality through a consistency loss $\\|\\Lambda^* - \\Lambda_i\\|^2$ and, in parallel, a physics residual loss that requires the predicted field to satisfy the recovered equation. The authors report that this alignment reduces the average nRMSE of long-trajectory predictions by about 30.22%, and on the out-of-distribution viscous Burgers task it beats plain auto-regressive finetuning by 37.8% across model sizes.","pith_inferences":["A testable extension is to apply the same coefficient-vector alignment to non-PDE auto-regressive models, such as any dynamical system with a parametric generative law; if the law is identifiable from a short window, consistency between windows should reduce compounding errors there as well.","The reported gains all involve finetuning pretrained checkpoints, so an untested but direct corollary of the paper's design is using PITA during pretraining itself, where the same alignment signal might shape the foundation model before downstream adaptation.","The method's reliance on a hand-built candidate library could be relaxed by generative or genetic library search; the paper's own ablation results suggest performance degrades gracefully when the library is incomplete, so an adaptive library might preserve gains without specialist tuning.","Because the overhead of PDE discovery is constant and independent of model size, the relative cost of PITA shrinks as models grow, making it a plausible plug-in for much larger PDE foundation models than the 500M-parameter case tested."],"forward_implications":["On long trajectories PITA reduces total nRMSE by about 30.22% on average across models and datasets, and the per-step error accumulation is visibly slowed in rolling-MSE plots.","The gains hold across model scales from 7M to 500M parameters and across diverse benchmarks, and the extra training cost does not grow with model size.","On out-of-distribution tasks, PITA improves accuracy even when the pretraining physics differs: about 5.93% average nRMSE reduction on compressible Navier-Stokes with different viscosities and about 37.8% on the unseen viscous Burgers equation.","PITA is data-efficient in downstream finetuning: from a pretrained checkpoint it reaches the same accuracy with roughly 500 samples that plain auto-regressive finetuning needs 1000 samples to reach.","The method remains competitive when the discovered physics is partly corrupted or the candidate library is incomplete; in ablations it degrades gracefully to around or above the auto-regressive baseline."],"supporting_citations":[{"why":"Supplies the sparse-regression formulation and finite-difference library construction used to discover governing equations from data.","marker":"[Rudy et al., 2017]"},{"why":"Provides the alternating-direction optimization strategy for coupling neural-network parameters with sparse PDE coefficients.","marker":"[Chen et al., 2021]"},{"why":"The DPOT auto-regressive pretraining method whose checkpoints are the main tested foundation models.","marker":"[Hao et al., 2024]"},{"why":"Supplies the MPP baseline and multi-physics pretraining checkpoints compared in the experiments.","marker":"[McCabe et al., 2023]"},{"why":"Defines the Fourier neural operator architecture used as an additional single-family baseline.","marker":"[Li et al., 2020b]"},{"why":"Provides the PDEBench datasets and the nRMSE evaluation convention used for most experiments.","marker":"[Takamoto et al., 2022]"},{"why":"Motivates the uncertainty-based weighting that balances data, physics, and consistency losses.","marker":"[Kendall et al., 2018]"},{"why":"Supplies the out-of-distribution viscous Burgers dataset used for the downstream generalization experiments.","marker":"[Boussif et al., 2022]"}],"fun_headline_variants":["Aligning discovered physics cuts PDE forecast error by 30%","No-prior physics alignment improves auto-regressive PDE forecasts","Self-supervised trick fixes auto-regressive PDE shortcut problem","PITA aligns physics across time, slices PDE forecast error 30%","Physics alignment without priors cuts auto-regressive PDE error 30%"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole physics signal rests on the assumption that sparse regression over a short, downsampled window—three temporal frames and a quarter of the spatial points—recovers a coefficient vector that is comparable to the vector from ten full-resolution ground-truth frames, and that the candidate library is normalized identically across windows.","fun_headline_variants_meta":{"raw":{"variants":["Aligning discovered physics cuts PDE forecast error by 30%","No-prior physics alignment improves auto-regressive PDE forecasts","Self-supervised trick fixes auto-regressive PDE shortcut problem","PITA aligns physics across time, slices PDE forecast error 30%","Physics alignment without priors cuts auto-regressive PDE error 30%"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000789,"raw_usage":{"total_tokens":3454,"prompt_tokens":896,"completion_tokens":2558,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":512,"completion_tokens_details":{"reasoning_tokens":2470}},"tokens_in":512,"tokens_out":2558,"duration_ms":15848,"temperature":1.0,"reasoning_tokens":2470,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:00:46.338289+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Finetune a PDE foundation model with PITA on a trajectory whose governing equation includes a term deliberately left out of the candidate library, for example a cubic damping term when the library only contains quadratic terms. If the consistency loss then forces the recovered coefficients toward a $\\Lambda^*$ that cannot represent the true law, and prediction error stays at or above the plain auto-regressive baseline, the reported gains rely on assuming the library contains the relevant terms rather than on the alignment mechanism itself.","supporting_citations":[],"review_version":1}