{"id":"304e56f3-c844-48a5-8d20-6c3c28fa4820","arxiv_id":"2505.08740","paper_version":3,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Supervising a neural operator's predicted Jacobian du/dp during training, called SC-FNO, sharply improves parameter inversion, sensitivity accuracy, and robustness to parameter shift compared to standard FNO and FNO with physics-informed regularization.","lead":"This paper adds a training penalty that forces a Fourier Neural Operator to match the true sensitivity of its predictions to physical parameters, alongside matching the solution itself. The result is better parameter recovery from observations, better robustness to out-of-range parameters, and accurate sensitivity estimates, at the cost of 30-130% more training time per epoch.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Jacobian labels for PDEs are unvalidated; SC-FNO's sensitivity and inversion gains may measure fit to solver gradients rather than true PDE sensitivities.","rationale":"The reader's weakest_assumption identifies exactly the most load-bearing concern. The sensitivity loss is the entire methodological contribution; if its supervision targets are wrong, the headline claims about sensitivity accuracy and the explanation for inversion gains collapse into 'the model matches the solver's Jacobian,' regardless of whether that Jacobian is the true PDE sensitivity. The concern is not a stylistic or reproducibility issue; it attacks the validity of the measured quantity. The inversion results are somewhat more robust to this concern because the synthetic observations and the labels come from the same solver, so matching solver Jacobians can still help recover the parameters that generated those observations. However, the scientific claim is about parametric differential equations, not about a particular discretization, so unvalidated Jacobian labels remain a genuine conditional on the central claim. I do not see a stronger objection: the method is internally consistent, the ODE validation is a positive control, and the AD-versus-FD comparison in Table 5 provides partial but not definitive evidence for PDE label quality. The absence of error bars and the abstract's overstated training-time claim are real weaknesses but secondary to the correctness of the supervision signal. Therefore the reader's CONDITIONAL verdict stands, and my read does not change it.","tokens_in":21525,"tokens_out":6554,"duration_ms":75321,"concrete_test":"For PDE1 and PDE2, recompute the supervision labels du/dp in three independent ways at matched discretizations: (i) the paper's torchdiffeq AD procedure; (ii) central finite differences of the same solver with a step-size sweep, using an optimal perturbation; and (iii) a higher-resolution reference obtained by doubling N_t and S_x and tightening solver tolerances, with sensitivities computed by an adjoint or complex-step method. Report the relative L2 differences between the three label sets. Then retrain SC-FNO using only the higher-resolution reference labels and rerun the core evaluations from Table 1: sensitivity R^2 for each parameter and multi-parameter inversion R^2. If the label sets differ by more than roughly 10%, or if the Table 1 entries shift materially, the reported gains depend on unvalidated solver Jacobians.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central intervention is the sensitivity loss L_s in Eq. (6), which supervises the model against 'true' Jacobians du/dp. For ODE1 those labels are checked against the analytical sensitivities in Eqs. (9)-(11), and Table D.13 reports excellent agreement. But for PDE1-PDE4 the paper never verifies the Jacobian labels: no grid-convergence study, no comparison against an independent adjoint or high-order finite-difference Jacobian, and no analytical fallback. The labels are whatever the torchdiffeq-based differentiable solver returns after AD through time-stepping, which for stiff, nonlinear, or bifurcating systems (e.g., Allen-Cahn, Navier-Stokes at Re=1000) can deviate from the continuum PDE sensitivity. Table 5's AD-vs-FD comparison for PDE1 is downstream model performance, not a direct label agreement check, so it does not resolve the concern. Because the inversion experiments generate their synthetic observations with the same solver, an SC-FNO that faithfully matches the solver's Jacobian will appear to invert parameters well even if that Jacobian is wrong relative to the true PDE. The reported sensitivity R^2 improvements in Tables 1-4 would then be measures of agreement with an unvalidated label, not with the actual parametric PDE. Thus the load-bearing premise of the central claim, that SC-FNO accurately estimates parameter sensitivities and that this accuracy drives the inversion gains, is not yet established for the PDE cases.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes Sensitivity-Constrained Fourier Neural Operators (SC-FNO), which augment the standard FNO data loss with a sensitivity loss L_s (Eq. 6). The loss penalizes the difference between the model's AD-computed Jacobian \\partial \\hat{u}/\\partial p and a 'true' Jacobian \\partial u/\\partial p prepared offline by a differentiable torchdiffeq-based solver or by finite differences. The authors evaluate four loss configurations on two ODEs and four PDEs, reporting that sensitivity supervision improves sensitivity R^2, parameter inversion accuracy, robustness to out-of-range parameter perturbations, and data efficiency relative to FNO and FNO-PINN, while adding 30-130% training time per epoch. The framework is also demonstrated on WNO, MWNO, DeepONet, and on an 82-parameter zoned Burgers equation. Section 3.5 and Appendix D.3 validate the Jacobian generation for ODE1 against an analytical solution, but not for the PDE cases.","tokens_in":21778,"tokens_out":4049,"duration_ms":41490,"significance":"If the reported gains are genuine, the contribution is significant and practical: a simple additive loss that makes neural operator surrogates usable for sensitivity analysis and gradient-based inversion, with apparent applicability across multiple operator architectures. The paper is commendably broad empirically, releases code, and includes a direct analytical validation for ODE1. Because the central mechanism is supervised learning of solver-provided Jacobians, however, the scientific weight of the PDE results hinges on the correctness of those Jacobian labels, and this is currently not established. The paper also provides machine-checkable pseudocode and reproducible experiment structure, which strengthens the presentation.","major_comments":[{"comment":"The Jacobian labels for PDE1-PDE4 are never validated. Table D.13 compares AD and finite-difference sensitivities only against the analytical solution of ODE1; no analogous check exists for any PDE. Since the sensitivity loss trains the model to reproduce exactly these labels, any solver error (discretization, AD through time-stepping, stiffness effects) is learned as ground truth, and the reported sensitivity R^2 improvements could measure agreement with an inaccurate label rather than with the true PDE sensitivity. Please add a grid-convergence study for \\partial u/\\partial p and a comparison against an independent adjoint or high-order finite-difference Jacobian for at least PDE1-PDE3.","section":"Section 2.3 and Appendix D.3"},{"comment":"The AD-versus-FD comparison for PDE1 reports downstream SC-FNO model performance, not agreement between the two Jacobian labeling methods. If both labels share a common discretization error, both trained models can perform well on the surrogate metrics while the underlying sensitivities are still wrong. Please report direct agreement statistics between the AD and FD Jacobians on the same PDE1 trajectories, such as per-parameter R^2 and relative L^2, before training any surrogate.","section":"Section 3.5 and Table 5"},{"comment":"The inversion experiments use synthetic observations generated by the same differentiable solver that produced the training labels. This creates a potential circularity: an SC-FNO that faithfully matches the solver's Jacobians can appear to invert parameters successfully even if those Jacobians deviate from the true PDE sensitivities. Please re-run at least one inversion benchmark with observations generated by an independent solver or a different discretization (or with the analytical solution in the ODE case) and confirm that the inversion gains persist.","section":"Section 3.1 and Figures 1-2"},{"comment":"All quantitative comparisons in the main tables are based on single training runs, without error bars or repeated seeds. The reported margins are often large, so seed variability may not overturn the main conclusion, but the claim that SC-FNO outperforms FNO and FNO-PINN would be more robust with mean and standard deviation over at least 3-5 seeds, particularly for the low-data regimes (N=100) and for the high-dimensional PDE2-zoned case.","section":"Tables 1-5 and Appendix D"}],"minor_comments":[{"comment":"The pseudocode states that c1, c2, and c3 are 'learnable coefficients', but the paper gives no details on how these coefficients are updated or which multi-task learning weighting scheme is used; please clarify or cite the method.","section":"Algorithms 2 and 3"},{"comment":"The sentence 'we randomly select a subset of spatial-temporal points in each epoch (n < Nspatial points × t < Ttime points)' is not a well-formed inequality; please rewrite it to define the sampled index sets clearly.","section":"Section 2.4"},{"comment":"The abstract claims the approach 'decreases training time while maintaining accuracy', but the reported per-epoch training times in Table C.8 all increase relative to FNO; the intended claim is presumably that fewer data or epochs are needed to reach a given accuracy, and this should be stated precisely.","section":"Abstract and Section 3.6"},{"comment":"The training-time measurements are reported without specifying the hardware and software versions; please state the computing environment used for the per-epoch times.","section":"Table C.8"},{"comment":"The DeepONet baselines show very low sensitivity R^2 values (for example, 0.117 for \\partial u/\\partial c in D.9), which raises the question of whether the DeepONet architecture and hyperparameters are comparable in capacity and training effort to the FNO baselines; please comment on this.","section":"Appendix D.1, Tables D.9-D.10"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about unvalidated PDE Jacobian labels is, on my reading, the key obstacle to accepting the paper's central claim. The analysis can be repaired within the manuscript's scope by adding direct Jacobian-label validation, cross-solver inversion tests, and repeated-seed statistics. I also note that this appears to be an ICLR 2025 conference paper; if the journal submission is intended to be an extended version, the authors should integrate the requested validation into the main text rather than leaving it as an appendix."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: this is a genuinely useful paper. The core idea is simple—add a loss term that supervises du/dp against Jacobians from a differentiable solver—and it delivers large, consistent gains across ODEs, five PDEs, and four operator families. The inversion improvements (R2 from ~0.65 to ~0.99, relative L2 down by 5–6x) are not subtle. The paper also shows that FNO's poor sensitivity estimates are a real mechanism behind its inversion failures, and that sensitivity supervision helps with data efficiency and out-of-range robustness. That is a concrete, transferable finding for anyone building surrogates for calibration or inverse problems.\n\nCredit where due: the authors cite the relevant gradient-enhancement literature (Sobolev training, gradient-enhanced NNs) and position their contribution accurately: applying this to neural operators with respect to physical parameters, in a time-integrated sense. They test the method on multiple architectures, not just FNO. They include a finite-difference variant, so the method is usable when differentiability is unavailable. The experiments are broad, and the observation that PINN loss does not fix sensitivities is a nice analytical point.\n\nSoft spots, in order of severity. First, the PDE Jacobian labels are not validated. The sensitivity loss supervises against solver AD Jacobians; the paper checks these against analytical gradients only for ODE1. For PDE1–PDE4, there is no grid-convergence check or comparison to an independent adjoint. This matters because if the solver Jacobians are off, the reported 'true sensitivity' R2 is just agreement with the solver. I would not call it fatal—for the synthetic inversion tasks, matching the solver is the relevant target—but it is a real gap and should be fixed with at least one PDE validation case.\n\nSecond, there are no error bars or multiple seeds anywhere. The performance gaps are large enough that the qualitative conclusions likely survive, but the paper would be much stronger with variance estimates, especially for the smaller-N runs.\n\nThird, the abstract says 'decreases training time' but Table C.8 shows SC-FNO costs 30–130% more per epoch at the same sample count. The claim is only true in the zoned high-dimensional comparison, where SC-FNO needs fewer samples. That is an overstatement and should be softened.\n\nThe citation pattern looks fine; no red flags. Overall, the central empirical finding holds up: sensitivity supervision is a cheap, architecture-agnostic way to make neural operators usable for inversion. The paper deserves serious referee time. My recommendation: engage with it, and require the label-validation check and error bars before final acceptance.","headline":"A practical, well-executed study showing that supervising parameter Jacobians makes FNOs dramatically better at inversion and robustness; the main gap is unvalidated PDE sensitivity labels, which is fixable.","tokens_in":22351,"tokens_out":2726,"would_cite":true,"duration_ms":27662,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that training Fourier Neural Operators with an additional loss on parameter sensitivities yields surrogates whose gradients are accurate and whose parameter-inversion performance is far better than plain FNO or FNO-PINN…","keywords":["Fourier Neural Operators","parameter inversion","sensitivity analysis","Jacobian supervision","parametric differential equations","physics-informed regularization","differentiable solvers","neural operators"],"falsifier":"Take one of the paper's PDE cases, say PDE2, and recompute the Jacobian labels with two independent methods, such as Richardson-extrapolated finite differences on the original solver and an adjoint solver; if SC-FNO's advantage over FNO in sensitivity $R^2$ or inversion $R^2$ largely disappears when the labels change, then the improvement is an artifact of the label-generation pipeline rather than of the sensitivity constraint itself.","tokens_in":21280,"feed_emoji":"🧮","tokens_out":11068,"duration_ms":92618,"temperature":0.7,"pith_summary":"The paper argues that a neural operator can learn solution paths $u(t)$ very accurately while still misrepresenting how those solutions depend on the parameters $p$, and that this hidden gradient error is what makes plain Fourier Neural Operators poor at inverse problems. To fix this, the authors add a sensitivity loss that compares the model's Jacobian $\\partial \\hat u/\\partial p$, obtained by automatic differentiation, with true Jacobians $\\partial u/\\partial p$ computed once from a differentiable solver or by finite differences. The resulting Sensitivity-Constrained FNO (SC-FNO) keeps forward accuracy and lifts sensitivity $R^2$ values from roughly 0.2-0.8 to 0.91-0.99 across ODEs, a damped wave equation, forced Burgers, Navier-Stokes vorticity, and Allen-Cahn test cases. It also improves multi-parameter inversion (for example, from $R^2=0.657$ to $R^2=0.987$ for one PDE1 parameter), works with up to 82 parameters, needs less training data, and stays accurate when test parameters are perturbed beyond the training range. A sympathetic reader would take this as evidence that explicit supervision of parameter sensitivities is a simple and broadly applicable regularizer for learned surrogate models.","feed_headline":"Gradient-aware training fixes neural PDE surrogates for inversion","feed_subtitle":"Teaching neural operators true parameter gradients cuts inversion error by up to 6x.","key_machinery":"The load-bearing object is the sensitivity loss $L_s = \\frac{1}{M}\\sum_{j=1}^{M} \\|\\partial \\hat u(x_j,t_j;p)/\\partial p - \\partial u(x_j,t_j;p)/\\partial p\\|^2$, where $\\partial \\hat u/\\partial p$ is the Jacobian of the network output with respect to the input parameters, obtained by automatic differentiation through the operator, and $\\partial u/\\partial p$ is a precomputed true Jacobian from a differentiable numerical solver or a fourth-order finite-difference approximation. The model itself is a standard Fourier Neural Operator, a network that learns maps between function spaces using learnable convolutions in the Fourier domain; the only change is that training minimizes the usual solution loss plus this Jacobian-matching term, with the equation-based PINN loss optional. The sensitivity term is what forces the network to use the parameters $p$ correctly, and the paper's experiments isolate it as the source of the inversion and extrapolation improvements.","core_discovery":"The central claim is that Fourier Neural Operators can match solution paths while badly mispredicting the parameter sensitivities $\\partial u/\\partial p$, and this sensitivity mismatch is the main reason they fail at parameter inversion and under extrapolated parameters. Adding the sensitivity loss $L_s$ fixes that: SC-FNO keeps solution accuracy comparable to FNO while raising sensitivity $R^2$ from about 0.2-0.8 to 0.91-0.99 in the paper's test beds, and in simultaneous multi-parameter inversion it raises $R^2$ from 0.657 to 0.987 for the parameter c of PDE1 and keeps all PDE2 parameters above 0.96 where FNO hovers near 0.85. The paper also reports that the physics-informed equation loss in FNO-PINN gives only small sensitivity gains, with $R^2$ below 0.52 for most PDE2 gradients, so the sensitivity term is doing the work. The same pattern holds for ODEs, for Navier-Stokes vorticity, for Allen-Cahn near bifurcation, for other neural operator architectures, and for sensitivity labels produced by finite differences rather than automatic differentiation.","pith_inferences":["If the paper is right, solution-path accuracy is a poor diagnostic for a surrogate's reliability in inverse or optimization settings; benchmarks for learned surrogates should report $\\partial u/\\partial p$ accuracy, because that is what inversion and data assimilation actually consume.","The archived Jacobian labels are exact in the paper, but the argument implies that approximate labels from coarser solvers or adjoint estimators might suffice to steer learning, which would remove the main data-preparation cost in practice.","The near-ineffectiveness of the PINN loss suggests a design rule worth testing: when the quantity of interest depends on parameters, supervise the parameter-gradient directly rather than relying on PDE residuals, which contain no information about $\\partial u/\\partial p$.","A natural extension is to apply the same constraint to sensitivities with respect to initial and boundary conditions; the ODE results already show gains for initial-condition sensitivities, so boundary-condition sensitivities are the obvious next test."],"forward_implications":["Surrogates trained this way can be placed inside gradient-based optimization and data-assimilation loops: backpropagating through SC-FNO recovers parameters with inversion $R^2$ mostly above 0.9 and relative $L^2$ errors roughly 1/6 to 1/2.8 those of FNO or FNO-PINN.","Sensitivity supervision substitutes for training data: with 100 to 500 samples SC-FNO keeps the accuracy that FNO reaches only with more data, and in the 82-parameter zoned Burgers test, SC-FNO trained on 100 samples beats FNO trained on 500.","The improvement transfers to other neural operators, including Wavelet, Multiwavelet, and DeepONet architectures, so the conclusion is about the loss function rather than about Fourier networks specifically.","Because finite-difference sensitivity labels work nearly as well as automatic differentiation labels, the method applies to existing simulation codes that are not differentiable.","Gradient-aware models tolerate concept drift: at 40% parameter perturbation beyond the training range, SC-FNO keeps solution $R^2$ near 0.91-0.93 where FNO drops to 0.53-0.73."],"supporting_citations":[{"why":"Introduces the Fourier Neural Operator architecture that the paper modifies and uses as the primary baseline.","marker":"Li et al. (2021)"},{"why":"Documents the difficulty FNOs have with parameter inversion and proposes physics-informed regularization, giving the FNO-PINN baseline the paper compares against.","marker":"Li et al. (2023)"},{"why":"Defines the physics-informed neural network equation loss used as the optional regularizer and comparison method.","marker":"Raissi et al. (2019)"},{"why":"Provides the Sobolev-training precedent for supervising neural networks with derivative information.","marker":"Czarnecki et al. (2017)"},{"why":"Supplies the adjoint/neural-ODE background for computing gradients through time-stepping solvers.","marker":"Chen et al. (2018)"},{"why":"Provides the Wavelet Neural Operator used to test whether the sensitivity constraint transfers to other architectures.","marker":"Tripura & Chakraborty (2023)"},{"why":"Provides the Multiwavelet Neural Operator used in the cross-architecture generalization experiments.","marker":"Gupta et al. (2021)"},{"why":"Provides the DeepONet architecture used to show the sensitivity loss helps beyond Fourier-based operators.","marker":"Wang et al. (2021)"}],"fun_headline_variants":["Sensitivity loss tames neural PDE surrogates for inverse problems","Teaching neural operators true gradients boosts inversion accuracy","Sensitivity-constrained FNOs beat standard FNO on parameter inversion","Fixing sensitivity estimates makes neural PDE surrogates invert well","Sensitivity regularization key for neural operators in inverse problems"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The training targets that teach the model its gradients are the Jacobians supplied by a differentiable solver or finite differences, and for the PDE cases the paper assumes these Jacobians are correct rather than verifying them against an independent analytical or high-accuracy reference.","fun_headline_variants_meta":{"raw":{"variants":["Sensitivity loss tames neural PDE surrogates for inverse problems","Teaching neural operators true gradients boosts inversion accuracy","Sensitivity-constrained FNOs beat standard FNO on parameter inversion","Fixing sensitivity estimates makes neural PDE surrogates invert well","Sensitivity regularization key for neural operators in inverse problems"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000529,"raw_usage":{"total_tokens":2557,"prompt_tokens":961,"completion_tokens":1596,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":577,"completion_tokens_details":{"reasoning_tokens":1513}},"tokens_in":577,"tokens_out":1596,"duration_ms":11594,"temperature":1.0,"reasoning_tokens":1513,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:47:39.267435+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take one of the paper's PDE cases, say PDE2, and recompute the Jacobian labels with two independent methods, such as Richardson-extrapolated finite differences on the original solver and an adjoint solver; if SC-FNO's advantage over FNO in sensitivity $R^2$ or inversion $R^2$ largely disappears when the labels change, then the improvement is an artifact of the label-generation pipeline rather than of the sensitivity constraint itself.","supporting_citations":[{"cited_title":"Multiwavelet-based Operator Learning for Differential Equations","cited_arxiv_id":null,"evidence_quote":"Provides the Multiwavelet Neural Operator used in the cross-architecture generalization experiments."}],"review_version":1}