Pith. sign in

REVIEW 3 major objections 4 minor 67 references

This paper claims that freezing a neural operator backbone and running a conditional diffusion model over its learned low-rank coefficient space produces a predictive distribution that approximates the true stochastic operator, improving di

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 04:42 UTC pith:T7PLSAHC

load-bearing objection Useful modular recipe, but the KL-coefficient story rests on a proof that optimizes the wrong objective; the diffusion head is a plausible empirical contribution that needs honest reframing. the 3 major comments →

arxiv 2602.04139 v2 pith:T7PLSAHC submitted 2026-02-04 cs.LG physics.comp-ph

Generative Neural Operators through Diffusion Last Layer

Sungwon Park , Anthony Zhou , Hongjoong Kim , Amir Barati Farimani This is my paper
classification cs.LG physics.comp-ph
keywords neural operatorsdiffusion modelsuncertainty quantificationKarhunen–Loève expansionstochastic PDEsgenerative surrogatesautoregressive rolloutoperator encoder
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper is trying to establish that a deterministic neural operator surrogate can be turned into a probabilistic one by attaching a 'diffusion last layer'—a diffusion model trained not on the full output field but on the coefficients of a low-rank, input-dependent basis. The authors argue this coefficient space behaves like a truncated Karhunen-Loève expansion of the conditional output distribution, so sampling coefficients and decoding them yields draws from the true stochastic operator. A sympathetic reader would care because the method is modular: it leaves the backbone unchanged, inherits its discretization invariance, and adds uncertainty estimation at modest cost. On stochastic PDE benchmarks the method is competitive with pixel-space and latent diffusion baselines, and on deterministic chaotic rollouts it improves stability and produces spread that tracks actual error.

Core claim

The paper introduces the diffusion last layer (DLL), a probabilistic output head that parameterizes the conditional output distribution through a rank-r expansion u ≈ ξ^T Φ(a), where a neural operator backbone produces input-dependent basis functions Φ(a) and a neural functional encoder produces coefficients ξ = NF(u). A conditional diffusion model, trained with velocity matching, generates new coefficients, and decoding through the frozen basis yields samples of the target field. Under universal approximation assumptions, the jointly trained operator encoder recovers the rank-r truncated Karhunen-Loève subspace of the conditional covariance of u given a (Proposition 4.1), and the diffusion

What carries the argument

The operator encoder: a pair of networks (NO(a) generating r basis functions, NF(u) generating r coefficients) trained jointly by an MSE reconstruction loss, so the reconstruction is the orthogonal projection onto the subspace spanned by the input-dependent basis. Under a universal approximation assumption, the optimizing subspace is exactly the rank-r truncated Karhunen-Loève subspace of the conditional covariance of u given a. The diffusion last layer then runs a conditional velocity-matching diffusion process in that r-dimensional coefficient space, trained with the velocity loss LV; decoding samples with the frozen basis produces the predictive field distribution.

Load-bearing premise

The load-bearing premise is that the jointly trained operator basis actually equals the optimal rank-r Karhunen-Loève subspace of the conditional output distribution for each input, and that rank r=64 captures the essential structure; the paper provides no per-condition check that the learned subspace is close to the conditional KL subspace.

What would settle it

For a fixed test input, draw many target realizations from the true stochastic operator, estimate the conditional covariance and its top-64 eigenspace numerically, then compare the DLL encoder's reconstruction error to the optimal rank-64 KL error. If the learned-basis error is substantially larger, or if the covariance eigengap at rank 64 is small, the 'diffusion in truncated KL coefficient space' claim fails and DLL is better described as an ad hoc latent diffusion.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • On stochastic Burgers' and Darcy flow, DLL improves energy distance and sliced Wasserstein metrics over deterministic FNO, FNO-dropout, and PNO, and is competitive with pixel-space and latent diffusion baselines.
  • On Kuramoto-Sivashinsky and Kolmogorov flow, DLL improves long-horizon rollout NRMSE and CRPS over the FNO backbone and keeps spread-to-skill ratio near 1, indicating calibrated uncertainty under compounding error.
  • The method preserves the backbone's discretization invariance because diffusion operates on a fixed small coefficient vector while the basis is generated by the operator itself.
  • Because the encoder is frozen after training, DLL adds uncertainty modeling to an already trained neural operator without retraining the backbone, at a parameter cost comparable to the backbone.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable implication of the paper's interpretation: for a fixed input, the learned basis should nearly match the empirical Karhunen-Loève eigenbasis of the conditional covariance; checking this per-input would show whether the 'truncated KL coefficient space' claim is literal or only an analogy.
  • The same coefficient-space diffusion could be applied to inverse problems, treating sampled coefficients as draws that propagate to a posterior over fields—an extension the paper notes but does not demonstrate.
  • The rollout-stability gains may come partly from diffusion acting as a stochastic regularizer; ablating the diffusion head with a simple Gaussian head over the same coefficients would separate generative benefit from coefficient-space smoothing.
  • Because the basis lives in the function space, DLL should extend to irregular geometries when paired with a geometry-aware operator backbone; testing this would show whether coefficient diffusion remains stable outside regular grids.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes the Diffusion Last Layer (DLL), a modular probabilistic head for neural operator backbones. DLL first trains an operator encoder that represents a target field u by a rank-r expansion u ≈ NF(u)^T NO(a), with an input-dependent basis Φ(a)=NO(a) and coefficients ξ=NF(u), minimizing the reconstruction loss in Eq. (5). The encoder is then frozen, and a conditional diffusion/flow-matching model is trained on the coefficient space R^r, conditioned on features of a. The theoretical narrative claims that minimizing Eq. (5) recovers the conditional Karhunen-Loève subspace and that, combined with the Wasserstein stability bound of Proposition 2.3, DLL parameterizes the conditional output distribution via diffusion in the truncated KL coefficient space. Experiments on stochastic Burgers and Darcy flow and on deterministic KS and Kolmogorov long-horizon rollouts compare DLL with FNO, MC-dropout FNO, PNO, pixel-space diffusion, and latent diffusion, reporting DLL as best or competitive on most distributional and rollout metrics.

Significance. If correct, the proposal is practically attractive: it is architecture-agnostic, adds uncertainty quantification to any neural operator backbone, and performs diffusion in a low-dimensional coefficient space rather than in pixel or function space. The paper also contains a complete and standard Wasserstein stability proof for velocity matching, careful functional-analytic lemmas in Appendix A.2, and detailed experimental configuration information, which are useful strengths. However, the central theoretical link to the Karhunen-Loève expansion is currently not established, because the reconstruction objective is uncentered while KL truncation is centered; the method as presented reduces to an ad hoc conditional latent diffusion on learned coordinates. The empirical claims are also weakened by the absence of repeated-seed error bars. The significance is therefore conditional: the engineering idea is plausible and the benchmarks are relevant, but the advertised interpretation and the strength of the empirical evidence need substantial revision.

major comments (3)
  1. [§4.1, Eq. (5); Proposition 4.1; Appendix A.2] Proposition 4.1 is not proved as stated. The proof minimizes L_OE(a)=E[||u-P_S u||^2|a] over r-dimensional subspaces S. The truncated Karhunen–Loève subspace, however, minimizes the centered objective E[||(u-μ(a))-P_S(u-μ(a))||^2|a], where μ(a)=E[u|a]. The two objectives differ by ||(I-P_S)μ(a)||^2. Whenever the conditional mean μ(a) is not contained in the top covariance subspace, the uncentered minimizer will spend basis capacity on representing μ(a), and the learned coefficients are not conditional KL coefficients. This is not an approximation-error issue; it holds even under Assumption A.11. Consequently, the §4.2 claim that DLL 'parameterizes the conditional output distribution via diffusion in the truncated KL coefficient space' is not justified by the supplied proof. The claim would become valid if the reconstruction included an explicit input-dependent mean term or the loss were
  2. [§5.2–§5.4, Tables 2–7] All empirical tables report single point estimates with no repeated-seed runs, confidence intervals, or significance tests. Several reported differences are small, e.g., DLL NRMSE 0.343 vs PNO 0.354 on KS (Table 4) and DM NRMSE 0.369 vs DLL 0.426 on Kolmogorov flow (Table 5). Without seed variability, the claims of 'best', 'second-best', and 'competitive' cannot be assessed reliably. I request at least 3–5 seeds with mean±std (or comparable intervals) for the main metrics, and related adjustments to the comparative wording.
  3. [Assumption A.11; §4.2] Assumption A.11 is very strong: it requires the neural networks to realize every r-dimensional subspace at fixed a and to realize the exact orthogonal projection onto any such basis. Even if granted as an idealization, the paper gives no per-condition or dataset-level check that the jointly trained FNO basis is close to the conditional covariance subspace, nor any diagnostic that r=64 is sufficient. Since the KL interpretation depends on this equivalence, the theoretical chain is fragile. I recommend either adding an explicit mean term or centered objective (which would make Proposition 4.1 valid under A.11) or providing empirical diagnostics comparing the learned basis to per-condition PCA/KL subspaces; absent that, the 'KL' interpretation should be dropped.
minor comments (4)
  1. [§3.3] The discussion linking non-zero velocity loss L_V(c) directly to epistemic uncertainty is heuristic. Residual training loss can also reflect model misspecification, optimization error, or aleatoric mismatch. Please soften or qualify this interpretation.
  2. [§2.2, Proposition 2.3] The bound is up to a Gronwall factor C that depends on the Lipschitz constant L(t) and is not quantified. The sentence 'small velocity matching error implies small transport error' should say 'up to this factor'. This is not a correctness issue but should be stated precisely.
  3. [References / formatting] There are several formatting/reference issues: 'Karhunen-Lo`eve' has a stray backtick, the reference 'Kohl et al., 24' should be 'Kohl et al., 2024', and some equations/rendered text throughout the appendix have spacing artifacts. Please proofread.
  4. [§5.4, Tables 6–7] The reconstruction comparison between AE and OE reports only NRMSE. Given the text's claims about compression-accuracy trade-offs, consider also reporting the compression ratio explicitly in the table caption and reporting seed variability here as well.

Circularity Check

1 steps flagged

No significant circularity in the empirical pipeline; the main caveat is Proposition 4.1's KL-subspace identification, which is a centered-vs-uncentered PCA proof gap rather than a by-construction circular reduction.

specific steps
  1. other [§4.1 Proposition 4.1; Appendix A.2 (proof of Prop. 4.1, Prop. A.10); §4.2]
    "Proposition 4.1 (Optimal rank-r reconstruction). Fix r∈N and a∈A. Under the assumptions in Appendix A.2, any global minimizer of LOE in (5) achieves inf LOE(a) = inf dim(S)=r E[∥u−P_Su∥2 |a], and the minimizing subspace is the rank-r truncated Karhunen–Loève subspace of the conditional covariance of u given a. … Minimizing (21) over ψ and using Assumption A.11 yields infψ,φ LOE(ψ,φ;a) = infdim(S)=r E[∥u−P_Su∥2 |a]."

    The proof of Proposition 4.1 only establishes optimality for the uncentered reconstruction loss E[∥u−P_Su∥²|a] used in (5) and (21), but the truncated Karhunen–Loève subspace in Appendix A.10 is defined as the minimizer of the centered objective E[∥(u−μ(a))−P_S(u−μ(a))∥²|a], with μ(a)=E[u|a]. These objectives differ by ∥(I−P_S)μ(a)∥², so the global minimizer of (5) need not be the conditional KL basis unless μ(a) lies in the chosen subspace. The proposition's KL conclusion therefore does not follow from the given proof, and the §4.2 claim that 'DLL parameterizes the conditional output distribution via diffusion in the truncated KL coefficient space' inherits that gap. This is a missing/non-sequitur proof step rather than a reduction of the result to its inputs, but it is load-bearing for t

full rationale

The empirical claims are self-contained against external data and benchmarks: stochastic Burgers and Darcy datasets are generated from the stated SPDE/PDE solvers, and KS/Kolmogorov rollouts follow APEBench; the reported ED, SWD, NRMSE, CRPS, and SSR metrics are held-out evaluation quantities, not objectives used to fit the diffusion velocity model. The velocity-matching loss (4) is a standard conditional flow-matching objective and is not tuned to the reported metrics, so there is no fitted-input-called-prediction pattern. Proposition 2.3 is an honest Wasserstein stability bound proved in Appendix A.1 via the continuity equation and Grönwall-type estimates, with explicit regularity assumptions; it does not assume its conclusion. Self-citations (e.g., Zhou et al., 2025a;b) appear only as related work on latent/physics diffusion and are not used to justify DLL's architecture or to import a uniqueness theorem. The only notable flaw is Proposition 4.1: its proof minimizes uncentered MSE while the KL subspace is defined through centered covariance, so the 'diffusion in KL coefficient space' interpretation is unestablished. This is a proof gap and correctness risk, not a circular reduction: the method and its benchmark results do not depend on the KL label, and no reported number is forced by construction. Overall circularity score is therefore low (2).

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 0 invented entities

The central method introduces no new physical entity. It relies on a handful of tuned hyperparameters (r=64, T=10, K=32) and on strong functional-approximation assumptions for the operator encoder. The diffusion and reconstruction components are standard building blocks, but the input-dependent basis is the paper's main design choice.

free parameters (5)
  • latent coefficient dimension r = 64
    Set to 64 for all benchmarks; the central representation capacity of DLL depends on this choice.
  • diffusion horizon T = 10
    Number of flow-matching steps used for DM, LDM, and DLL; affects sampling fidelity.
  • evaluation ensemble size K = 32
    Distributional metrics and uncertainty estimates are computed from 32 samples; results depend on this finite-sample choice.
  • noise schedule a_t, b_t = not specified
    The forward process (Eq. 3) requires a schedule; the paper states differentiable scheduling functions but never gives their form.
  • baseline Monte Carlo dropout rate p and ensemble size = p=0.2, K=32
    FNO-d comparison depends on these settings; they are chosen by the authors.
axioms (5)
  • ad hoc to paper Assumption A.11: the neural operator basis NO_ψ can realize every r-dimensional subspace of U at fixed a, and NF_φ can realize the orthogonal projection onto any such basis.
    Introduced in Appendix A.2 to make Proposition 4.1 true; in practice FNO/NF with fixed width cannot guarantee this.
  • standard math Assumption A.1: integrability and Lipschitz regularity of velocity fields along the flow.
    Technical regularity needed for Prop 2.3; standard in continuity-equation stability.
  • domain assumption Assumption A.4: learned velocity field v_ϕ is Lipschitz in the condition c.
    Used in Prop A.5 to connect nearby conditions to nearby output distributions; an empirical property, not verified.
  • domain assumption Ground-truth stochastic operator G‡ exists and the dataset pairs are generated from it (Eq. 1).
    The benchmark construction assumes numerical solvers produce faithful samples; standard but unverified.
  • domain assumption The discretization-invariance of FNO transfers to the DLL head; no multi-resolution experiments are performed.
    Claims of preserving discretization invariance rely on the backbone but are not tested with DLL at different resolutions.

pith-pipeline@v1.3.0-alltime-deepseek · 22999 in / 12903 out tokens · 132753 ms · 2026-08-03T04:42:32.076814+00:00 · methodology

0 comments
read the original abstract

Neural operators provide a powerful framework for learning discretization invariant mappings between function spaces, but standard deterministic models do not capture predictive uncertainty. We introduce diffusion last layer (DLL), a modular probabilistic output head for neural operator backbones. DLL represents target fields through an input dependent low rank expansion inspired by the Karhunen-Lo\'eve expansion and learns a conditional diffusion model over the corresponding coefficient space. This design enables efficient distributional modeling while preserving the structural advantages of operator learning. On stochastic PDE benchmarks with random forcing, DLL achieves strong distributional fidelity and performs competitively with pixel space and conventional latent diffusion baselines. In deterministic long horizon rollout tasks, DLL improves rollout stability over the underlying backbone and provides useful estimates of predictive uncertainty under compounding autoregressive errors. These results suggest that diffusion modeling in learned coefficient spaces offers a practical route to uncertainty aware neural operators.

Figures

Figures reproduced from arXiv: 2602.04139 by Amir Barati Farimani, Anthony Zhou, Hongjoong Kim, Sungwon Park.

Figure 1
Figure 1. Figure 1: Deterministic vs. generative neural operators. A standard neural operator (top) maps an input field a ∈ A to a single pre￾diction u ∈ U. Our Diffusion Last Layer (bottom) turns the same backbone into a conditional generator by attaching a lightweight diffusion head, producing a full predictive distribution p(u | a) over functions rather than a point estimate. metric and irregular domains (Li et al., 2023a;… view at source ↗
Figure 2
Figure 2. Figure 2: Training and inference pipeline for the Diffusion Last Layer (DLL). (1) Operator encoder. A NO backbone produces basis functions Φ(a) and a NF maps targets to coefficients ξ = NF(u), yielding the low-rank reconstruction uˆ = ξ ⊤Φ(a). (2) DLL training. With the encoder frozen, we train a conditional diffusion model in coefficient space: an MLP denoiser predicts noise (or score) from (a, xt, t) using NO feat… view at source ↗
Figure 3
Figure 3. Figure 3: Stochastic Burgers’ equation. Columns compare the ground truth and different surrogate models. For each method, the top panel shows multiple realizations of the solution field u(x) for a fixed input, illustrating sample diversity. The bottom panel shows the predictive mean (solid line) and an uncertainty band given by standard deviation (shaded region) estimated from the samples. C. Qualitative Results In … view at source ↗
Figure 4
Figure 4. Figure 4: Stochastic Darcy flow. For each method, we generate conditional samples of the solution field and summarize them by the sample mean (top) and per-pixel sample standard deviation (bottom). This visualization highlights both accuracy of the central prediction and the spatial structure of predictive uncertainty. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: KS equation. Long-horizon rollout comparison at rollout step 50. The black curve denotes the ground-truth solution, while the blue curve shows the predictive mean of each method. Shaded regions indicate predictive standard deviation estimated from generated samples. 24 [PITH_FULL_IMAGE:figures/full_fig_p024_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Kolmogorov flow. Rollout evaluation at the 50th step. Rows show the predictive mean (top), predictive standard deviation (middle), and the pointwise absolute error (bottom) for a representative test case across baselines and DLL. 25 [PITH_FULL_IMAGE:figures/full_fig_p025_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Rollout evaluation on KS (top row) and Kolmogorov flow (bottom row). We report NRMSE (left), CRPS (middle), and SSR (right) as a function of rollout step for baselines and DLL, illustrating long-horizon accuracy and uncertainty calibration. 26 [PITH_FULL_IMAGE:figures/full_fig_p026_7.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

67 extracted references · 4 linked inside Pith

  1. [1]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION format.date year duplicate empty "emp...

  2. [2]

    Gradient flows: in metric spaces and in the space of probability measures

    Ambrosio, L., Gigli, N., and Savar \'e , G. Gradient flows: in metric spaces and in the space of probability measures. Springer, 2005

    2005

  3. [3]

    Neural operators for accelerating scientific simulations and design

    Azizzadenesheli, K., Kovachki, N., Li, Z., Liu-Schiaffini, M., Kossaifi, J., and Anandkumar, A. Neural operators for accelerating scientific simulations and design. Nature Reviews Physics, 6 0 (5): 0 320--328, 2024

    2024

  4. [4]

    Weight uncertainty in neural network

    Blundell, C., Cornebise, J., Kavukcuoglu, K., and Wierstra, D. Weight uncertainty in neural network. In International conference on machine learning, pp.\ 1613--1622. PMLR, 2015

    2015

  5. [5]

    Spherical fourier neural operators: Learning stable dynamics on the sphere

    Bonev, B., Kurth, T., Hundt, C., Pathak, J., Baust, M., Kashinath, K., and Anandkumar, A. Spherical fourier neural operators: Learning stable dynamics on the sphere. In International conference on machine learning, pp.\ 2806--2823. PMLR, 2023

    2023

  6. [6]

    Why diffusion models don t memorize: The role of implicit dynamical regularization in training

    Bonnaire, T., Urfin, R., Biroli, G., and Mezard, M. Why diffusion models don t memorize: The role of implicit dynamical regularization in training. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

    2025

  7. [7]

    Probabilistic neural operators for functional uncertainty quantification

    B \"u lte, C., Scholl, P., and Kutyniok, G. Probabilistic neural operators for functional uncertainty quantification. Transactions on Machine Learning Research, 2025. ISSN 2835-8856

    2025

  8. [8]

    S., Boffi, N

    Chen, Y., Goldstein, M., Hua, M., Albergo, M. S., Boffi, N. M., and Vanden-Eijnden, E. Probabilistic forecasting with stochastic interpolants and föllmer processes. In Proceedings of the 41st International Conference on Machine Learning, volume 235. PMLR, 2024

    2024

  9. [9]

    Hyperdiffusion: Generating implicit neural fields with weight-space diffusion

    Erko c , Z., Ma, F., Shan, Q., Nie ner, M., and Dai, A. Hyperdiffusion: Generating implicit neural fields with weight-space diffusion. In Proceedings of the IEEE/CVF international conference on computer vision, pp.\ 14300--14310, 2023

    2023

  10. [10]

    and Ghahramani, Z

    Gal, Y. and Ghahramani, Z. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pp.\ 1050--1059. PMLR, 2016

    2016

  11. [11]

    P., and Salimans, T

    Gao, R., Hoogeboom, E., Heek, J., De Bortoli, V., Murphy, K. P., and Salimans, T. Diffusion models and gaussian flow matching: Two sides of the same coin. In The Fourth Blogpost Track at ICLR 2025, 2025

    2025

  12. [12]

    Guo, C., Pleiss, G., Sun, Y., and Weinberger, K. Q. On calibration of modern neural networks. In International conference on machine learning, pp.\ 1321--1330. PMLR, 2017

    2017

  13. [13]

    Gnot: A general neural operator transformer for operator learning

    Hao, Z., Wang, Z., Su, H., Ying, C., Dong, Y., Liu, S., Cheng, Z., Song, J., and Zhu, J. Gnot: A general neural operator transformer for operator learning. In International Conference on Machine Learning, pp.\ 12556--12569. PMLR, 2023

    2023

  14. [14]

    Dpot: Auto-regressive denoising operator transformer for large-scale pde pre-training

    Hao, Z., Su, C., Liu, S., Berner, J., Ying, C., Su, H., Anandkumar, A., Song, J., and Zhu, J. Dpot: Auto-regressive denoising operator transformer for large-scale pde pre-training. arXiv preprint arXiv:2403.03542, 2024

    Pith/arXiv arXiv 2024

  15. [15]

    Variational bayesian last layers

    Harrison, J., Willes, J., and Snoek, J. Variational bayesian last layers. In The Twelfth International Conference on Learning Representations, 2024

    2024

  16. [16]

    Poseidon: Efficient foundation models for pdes

    Herde, M., Raonic, B., Rohner, T., K \"a ppeli, R., Molinaro, R., de B \'e zenac, E., and Mishra, S. Poseidon: Efficient foundation models for pdes. Advances in Neural Information Processing Systems, 37: 0 72525--72624, 2024

    2024

  17. [17]

    Denoising diffusion probabilistic models

    Ho, J., Jain, A., and Abbeel, P. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33: 0 6840--6851, 2020

    2020

  18. [18]

    Huang, J., Yang, G., Wang, Z., and Park, J. J. Diffusionpde: Generative pde-solving under partial observation. Advances in Neural Information Processing Systems, 37: 0 130291--130323, 2024

    2024

  19. [19]

    P., and Mallat, S

    Kadkhodaie, Z., Guth, F., Simoncelli, E. P., and Mallat, S. Generalization in diffusion models arises from geometry-adaptive harmonic representations. In The Twelfth International Conference on Learning Representations, 2024. URL https://openreview.net/forum?id=ANvmVS2Yr0

    2024

  20. [20]

    Diffusion generative models in infinite dimensions

    Kerrigan, G., Ley, J., and Smyth, P. Diffusion generative models in infinite dimensions. In International Conference on Artificial Intelligence and Statistics, pp.\ 9538--9563. PMLR, 2023

    2023

  21. [21]

    Functional flow matching

    Kerrigan, G., Migliorini, G., and Smyth, P. Functional flow matching. In Proceedings of The 27th International Conference on Artificial Intelligence and Statistics, pp.\ 3934--3942, 2024

    2024

  22. [22]

    Apebench: A benchmark for autoregressive neural emulators of pdes

    Koehler, F., Niedermayr, S., Thuerey, N., et al. Apebench: A benchmark for autoregressive neural emulators of pdes. Advances in Neural Information Processing Systems, 37: 0 120252--120310, 2024

    2024

  23. [23]

    Benchmarking autoregressive conditional diffusion models for turbulent flow simulation

    Kohl, G., Chen, L., and Thuerey, N. Benchmarking autoregressive conditional diffusion models for turbulent flow simulation. In ICML 2024 AI for Science Workshop, 24

    2024

  24. [24]

    Tabddpm: Modelling tabular data with diffusion models

    Kotelnikov, A., Baranchuk, D., Rubachev, I., and Babenko, A. Tabddpm: Modelling tabular data with diffusion models. In International conference on machine learning, pp.\ 17564--17579. PMLR, 2023

    2023

  25. [25]

    Neural operator: Learning maps between function spaces with applications to pdes

    Kovachki, N., Li, Z., Liu, B., Azizzadenesheli, K., Bhattacharya, K., Stuart, A., and Anandkumar, A. Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research, 24 0 (89): 0 1--97, 2023

    2023

  26. [26]

    Being bayesian, even just a bit, fixes overconfidence in relu networks

    Kristiadi, A., Hein, M., and Hennig, P. Being bayesian, even just a bit, fixes overconfidence in relu networks. In International conference on machine learning, pp.\ 5436--5446. PMLR, 2020

    2020

  27. [27]

    Accurate uncertainties for deep learning using calibrated regression

    Kuleshov, V., Fenner, N., and Ermon, S. Accurate uncertainties for deep learning using calibrated regression. In International conference on machine learning, pp.\ 2796--2804. PMLR, 2018

    2018

  28. [28]

    Score-based generative modeling secretly minimizes the wasserstein distance

    Kwon, D., Fan, Y., and Lee, K. Score-based generative modeling secretly minimizes the wasserstein distance. Advances in Neural Information Processing Systems, 35: 0 20205--20217, 2022

    2022

  29. [29]

    The principles of diffusion models

    Lai, C.-H., Song, Y., Kim, D., Mitsufuji, Y., and Ermon, S. The principles of diffusion models. arXiv preprint arXiv:2510.21890, 2025

    Pith/arXiv arXiv 2025

  30. [30]

    Simple and scalable predictive uncertainty estimation using deep ensembles

    Lakshminarayanan, B., Pritzel, A., and Blundell, C. Simple and scalable predictive uncertainty estimation using deep ensembles. Advances in neural information processing systems, 30, 2017

    2017

  31. [31]

    Autoregressive image generation without vector quantization

    Li, T., Tian, Y., Li, H., Deng, M., and He, K. Autoregressive image generation without vector quantization. Advances in Neural Information Processing Systems, 37: 0 56424--56445, 2024

    2024

  32. [32]

    Fourier neural operator for parametric partial differential equations

    Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. Fourier neural operator for parametric partial differential equations. arXiv preprint arXiv:2010.08895, 2020

    Pith/arXiv arXiv 2010

  33. [33]

    Z., Liu, B., and Anandkumar, A

    Li, Z., Huang, D. Z., Liu, B., and Anandkumar, A. Fourier neural operator with learned deformations for pdes on general geometries. Journal of Machine Learning Research, 24 0 (388): 0 1--26, 2023 a

    2023

  34. [34]

    A., Stadler, M., Hundt, C., Azizzadenesheli, K., et al

    Li, Z., Kovachki, N., Choy, C., Li, B., Kossaifi, J., Otta, S., Nabian, M. A., Stadler, M., Hundt, C., Azizzadenesheli, K., et al. Geometry-informed neural operator for large-scale 3d pdes. Advances in Neural Information Processing Systems, 36: 0 35836--35854, 2023 b

    2023

  35. [35]

    Li, Z., Meidani, K., and Farimani, A. B. Transformer for partial differential equations’ operator learning. Transactions on Machine Learning Research, 2023 c

    2023

  36. [36]

    H., Kovachki, N

    Lim, J. H., Kovachki, N. B., Baptista, R., Beckham, C., Azizzadenesheli, K., Kossaifi, J., Voleti, V., Song, J., Kreis, K., Kautz, J., et al. Score-based diffusion models in function space. Journal of Machine Learning Research, 26 0 (158): 0 1--62, 2025

    2025

  37. [37]

    B., Byun, T., Kang, T., Kim, S., Lee, K., and Choi, S

    Lim, S., Yoon, E. B., Byun, T., Kang, T., Kim, S., Lee, K., and Choi, S. Score-based generative modeling through stochastic evolution equations in hilbert spaces. Advances in neural information processing systems, 36: 0 37799--37812, 2023

    2023

  38. [38]

    B-deeponet: An enhanced bayesian deeponet for solving noisy parametric pdes using accelerated replica exchange sgld

    Lin, G., Moya, C., and Zhang, Z. B-deeponet: An enhanced bayesian deeponet for solving noisy parametric pdes using accelerated replica exchange sgld. Journal of Computational Physics, 473: 0 111713, 2023

    2023

  39. [39]

    Lipman, Y., Chen, R. T. Q., Ben-Hamu, H., Nickel, M., and Le, M. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2023

    2023

  40. [40]

    Pde-refiner: Achieving accurate long rollouts with neural pde solvers

    Lippe, P., Veeling, B., Perdikaris, P., Turner, R., and Brandstetter, J. Pde-refiner: Achieving accurate long rollouts with neural pde solvers. Advances in Neural Information Processing Systems, 36: 0 67398--67433, 2023

    2023

  41. [41]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Liu, X., Gong, C., and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. In The Eleventh International Conference on Learning Representations, 2023

    2023

  42. [42]

    Lu, L., Jin, P., Pang, G., Zhang, Z., and Karniadakis, G. E. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature machine intelligence, 3 0 (3): 0 218--229, 2021

    2021

  43. [43]

    Calibrated uncertainty quantification for operator learning via conformal prediction

    Ma, Z., Pitt, D., Azizzadenesheli, K., and Anandkumar, A. Calibrated uncertainty quantification for operator learning via conformal prediction. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id=cGpegxy12T

    2024

  44. [44]

    Approximate bayesian neural operators: Uncertainty quantification for parametric PDE s

    Magnani, E., Kr \"a mer, N., Eschenhagen, R., Rosasco, L., and Hennig, P. Approximate bayesian neural operators: Uncertainty quantification for parametric PDE s. Transactions on Machine Learning Research, 2025 a . ISSN 2835-8856

    2025

  45. [45]

    Linearization turns neural operators into function-valued gaussian processes

    Magnani, E., Pf \"o rtner, M., Weber, T., and Hennig, P. Linearization turns neural operators into function-valued gaussian processes. In Forty-second International Conference on Machine Learning, 2025 b

    2025

  46. [46]

    Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators

    Pathak, J., Subramanian, S., Harrington, P., Raja, S., Chattopadhyay, A., Mardani, M., Kurth, T., Hall, D., Li, Z., Azizzadenesheli, K., et al. Fourcastnet: A global data-driven high-resolution weather model using adaptive fourier neural operators. arXiv preprint arXiv:2202.11214, 2022

    Pith/arXiv arXiv 2022

  47. [47]

    R., El-Kadi, A., Masters, D., Ewalds, T., Stott, J., Mohamed, S., Battaglia, P., et al

    Price, I., Sanchez-Gonzalez, A., Alet, F., Andersson, T. R., El-Kadi, A., Masters, D., Ewalds, T., Stott, J., Mohamed, S., Battaglia, P., et al. Probabilistic weather forecasting with machine learning. Nature, 637 0 (8044): 0 84--90, 2025

    2025

  48. [48]

    F., Meng, X., Zou, Z., Guo, L., and Karniadakis, G

    Psaros, A. F., Meng, X., Zou, Z., Guo, L., and Karniadakis, G. E. Uncertainty quantification in scientific machine learning: Methods, metrics, and comparisons. Journal of Computational Physics, 477: 0 111902, 2023

    2023

  49. [49]

    A., Florez, M

    Rahman, M. A., Florez, M. A., Anandkumar, A., Ross, Z. E., and Azizzadenesheli, K. Generative adversarial neural operators. Transactions on Machine Learning Research, 2022. ISSN 2835-8856. URL https://openreview.net/forum?id=X1VzbBU6xZ

    2022

  50. [50]

    A., Ross, Z

    Rahman, M. A., Ross, Z. E., and Azizzadenesheli, K. U- NO : U-shaped neural operators. Transactions on Machine Learning Research, 2023. ISSN 2835-8856

    2023

  51. [51]

    and Louppe, G

    Rozet, F. and Louppe, G. Score-based data assimilation. Advances in Neural Information Processing Systems, 36: 0 40521--40541, 2023

    2023

  52. [52]

    Lost in latent space: An empirical study of latent diffusion models for physics emulation

    Rozet, F., Ohana, R., McCabe, M., Louppe, G., Lanusse, F., and Ho, S. Lost in latent space: An empirical study of latent diffusion models for physics emulation. arXiv preprint arXiv:2507.02608, 2025

    arXiv 2025

  53. [53]

    Neural stochastic pdes: Resolution-invariant learning of continuous spatiotemporal dynamics

    Salvi, C., Lemercier, M., and Gerasimovics, A. Neural stochastic pdes: Resolution-invariant learning of continuous spatiotemporal dynamics. Advances in Neural Information Processing Systems, 35: 0 1333--1344, 2022

    2022

  54. [54]

    E., Asimaki, D., and Azizzadenesheli, K

    Shi, Y., Ross, Z. E., Asimaki, D., and Azizzadenesheli, K. Stochastic process learning via operator flow matching. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025

    2025

  55. [55]

    Shu, D., Li, Z., and Farimani, A. B. A physics-informed diffusion model for high-fidelity flow field reconstruction. Journal of Computational Physics, 478: 0 111972, 2023

    2023

  56. [56]

    M., Turner, R., and Mathieu, E

    Shysheya, A., Diaconu, C., Bergamin, F., Perdikaris, P., Hern \'a ndez-Lobato, J. M., Turner, R., and Mathieu, E. On conditional diffusion models for pde simulations. Advances in Neural Information Processing Systems, 37: 0 23246--23300, 2024

    2024

  57. [57]

    Deep unsupervised learning using nonequilibrium thermodynamics

    Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning, pp.\ 2256--2265. pmlr, 2015

    2015

  58. [58]

    Selective underfitting in diffusion models

    Song, K., Kim, J., Chen, S., Du, Y., Kakade, S., and Sitzmann, V. Selective underfitting in diffusion models. arXiv preprint arXiv:2510.01378, 2025

    arXiv 2025

  59. [59]

    P., Kumar, A., Ermon, S., and Poole, B

    Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021

    2021

  60. [60]

    Tran, A., Mathews, A., Xie, L., and Ong, C. S. Factorized fourier neural operators. In The Eleventh International Conference on Learning Representations, 2023

    2023

  61. [61]

    Geofunflow: Geometric function flow matching for inverse operator learning over complex geometries

    Wang, S., Wu, Z., van Dijk, D., and Lu, L. Geofunflow: Geometric function flow matching for inverse operator learning over complex geometries. arXiv preprint arXiv:2509.24117, 2025

    arXiv 2025

  62. [62]

    A., Klink, P., Pajarinen, J., and Peters, J

    Watson, J., Lin, J. A., Klink, P., Pajarinen, J., and Peters, J. Latent derivative bayesian last layer networks. In International Conference on Artificial Intelligence and Statistics, pp.\ 1198--1206. PMLR, 2021

    2021

  63. [63]

    Uncertainty quantification for fourier neural operators

    Weber, T., Magnani, E., Pf \"o rtner, M., and Hennig, P. Uncertainty quantification for fourier neural operators. In ICLR 2024 workshop on AI4DifferentialEquations in science, 2024

    2024

  64. [64]

    Weight diffusion for future: Learn to generalize in non-stationary environments

    Xie, M., Li, S., Xie, B., Liu, C., Liang, J., Sun, Z., Feng, K., and Zhu, C. Weight diffusion for future: Learn to generalize in non-stationary environments. Advances in Neural Information Processing Systems, 37: 0 6367--6392, 2024

    2024

  65. [65]

    Numerical methods for stochastic computations: a spectral method approach

    Xiu, D. Numerical methods for stochastic computations: a spectral method approach. Princeton university press, 2010

    2010

  66. [66]

    Zhou, A., Li, Z., Schneier, M., Jr, J. R. B., and Farimani, A. B. Text2 PDE : Latent diffusion models for accessible physics simulation. In The Thirteenth International Conference on Learning Representations, 2025 a

    2025

  67. [67]

    Zhou, A., Wikner, A., Lancelin, A., Hassanzadeh, P., and Farimani, A. B. Reframing generative models for physical systems using stochastic interpolants. arXiv preprint arXiv:2509.26282, 2025 b

    arXiv 2025