pith. sign in

arxiv: 2601.07384 · v1 · submitted 2026-01-12 · 💻 cs.LG

CompNO: A Novel Foundation Model approach for solving Partial Differential Equations

Hamda Hmida , Hsiu-Wen Chang Joly , Youssef Mesri This is my paper

Pith reviewed 2026-05-16 14:48 UTC · model grok-4.3

classification 💻 cs.LG
keywords compositional neural operatorsparametric PDEsFourier neural operatorsfoundation modelsboundary condition enforcementoperator learningPDE solving
0
0 comments X

The pith

Compositional Neural Operators solve parametric PDEs by assembling specialized blocks for each differential operator instead of training one large model.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes Compositional Neural Operators (CompNO) as a way to create foundation models for solving partial differential equations. Rather than pretraining a monolithic architecture on diverse data, it first trains a library of separate Foundation Blocks, each a Fourier neural operator tuned to one basic operator such as convection or diffusion. These blocks are then combined using lightweight Adaptation Blocks to form solvers for specific PDEs, with an additional operator to enforce boundary conditions exactly. Tests on one-dimensional convection, diffusion, and Burgers' equations show lower errors than baselines on linear cases and comparable results on nonlinear ones, plus perfect boundary adherence.

Core claim

CompNO learns a library of Foundation Blocks where each block is a parametric Fourier neural operator specialized to a fundamental differential operator, assembles them via lightweight Adaptation Blocks into task-specific solvers that approximate the temporal evolution operator for target PDEs, and employs a dedicated boundary-condition operator to enforce Dirichlet constraints exactly at inference time.

What carries the argument

Library of Foundation Blocks, each a parametric Fourier neural operator specialized to one differential operator (convection, diffusion, nonlinear convection), assembled via Adaptation Blocks to approximate full PDE evolution operators.

If this is right

  • Lower relative L2 error than PFNO, PDEFormer, and in-context learning models on linear parametric systems.
  • Competitive performance with baselines on nonlinear Burgers' flows.
  • Exact satisfaction of boundary conditions with zero loss at domain boundaries.
  • Robust generalization across a range of Peclet and Reynolds numbers.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Reusing the same operator blocks across different PDEs could reduce the need for full retraining when new equations are encountered.
  • The modular design may make it easier to inspect which physical processes the model is emphasizing in its solution.
  • Expanding the block library could extend the approach to systems of coupled PDEs or higher-dimensional problems.
  • The exact boundary enforcement might eliminate the need for penalty terms in the loss function during training.

Load-bearing premise

That blocks specialized to single differential operators can be combined through lightweight adaptation without losing accuracy or introducing interference when approximating the full evolution of arbitrary target PDEs.

What would settle it

Training the blocks on isolated operators and then testing the assembled model on a PDE whose solution depends on strong interactions between operators not captured during individual block training; if relative L2 error rises above monolithic baselines, the compositional claim fails.

Figures

Figures reproduced from arXiv: 2601.07384 by Hamda Hmida, Hsiu-Wen Chang Joly, Youssef Mesri.

Figure 1
Figure 1. Figure 1: Illustration of CompNO architecture. The input consists of the initial function state F(x, t0), the physical parameter vector γ (e.g., velocity β, viscosity ν), and the Boundary Conditions (BCs). The Foundation Blocks are pre-trained Neural Operators that independently predict the time-evolution corresponding to specific elementary operators (e.g., ∇u, ∆u). The Aggregator is a neural network (linear or non… view at source ↗
Figure 2
Figure 2. Figure 2: The graph illustrates the extrapolation of convection equation. [PITH_FULL_IMAGE:figures/full_fig_p013_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: L2 relative error measured for different values of Pe and Re numbers. [PITH_FULL_IMAGE:figures/full_fig_p015_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Visualization of the model prediction with and without BCs. [PITH_FULL_IMAGE:figures/full_fig_p016_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Comparison between the Convection-Diffusion solutions and predictions for various [PITH_FULL_IMAGE:figures/full_fig_p017_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Comparison between the Burgers’ solutions and predictions for various Re at different [PITH_FULL_IMAGE:figures/full_fig_p018_6.png] view at source ↗
read the original abstract

Partial differential equations (PDEs) govern a wide range of physical phenomena, but their numerical solution remains computationally demanding, especially when repeated simulations are required across many parameter settings. Recent Scientific Foundation Models (SFMs) aim to alleviate this cost by learning universal surrogates from large collections of simulated systems, yet they typically rely on monolithic architectures with limited interpretability and high pretraining expense. In this work we introduce Compositional Neural Operators (CompNO), a compositional neural operator framework for parametric PDEs. Instead of pretraining a single large model on heterogeneous data, CompNO first learns a library of Foundation Blocks, where each block is a parametric Fourier neural operator specialized to a fundamental differential operator (e.g. convection, diffusion, nonlinear convection). These blocks are then assembled, via lightweight Adaptation Blocks, into task-specific solvers that approximate the temporal evolution operator for target PDEs. A dedicated boundary-condition operator further enforces Dirichlet constraints exactly at inference time. We validate CompNO on one-dimensional convection, diffusion, convection--diffusion and Burgers' equations from the PDEBench suite. The proposed framework achieves lower relative L2 error than strong baselines (PFNO, PDEFormer and in-context learning based models) on linear parametric systems, while remaining competitive on nonlinear Burgers' flows. The model maintains exact boundary satisfaction with zero loss at domain boundaries, and exhibits robust generalization across a broad range of Peclet and Reynolds numbers. These results demonstrate that compositional neural operators provide a scalable and physically interpretable pathway towards foundation models for PDEs.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 1 minor

Summary. The manuscript introduces Compositional Neural Operators (CompNO), a framework that pretrains a library of Foundation Blocks (parametric FNOs specialized to individual operators such as convection or diffusion) and assembles them via lightweight Adaptation Blocks into solvers for target parametric PDEs, augmented by a dedicated boundary-condition operator that enforces exact Dirichlet conditions at inference. Validation is performed on 1D convection, diffusion, convection-diffusion, and Burgers' equations from the PDEBench suite, with the central claims being lower relative L2 error than PFNO, PDEFormer, and in-context learning baselines on linear cases, competitive performance on nonlinear Burgers' flows, exact boundary satisfaction (zero loss at boundaries), and robust generalization across Peclet and Reynolds numbers.

Significance. If the performance and composability claims hold with adequate verification, the work would offer a more modular and interpretable alternative to monolithic scientific foundation models for PDEs, potentially reducing pretraining costs by reusing specialized operator blocks and providing a pathway to physically grounded composition. The exact boundary enforcement and parameter-range robustness would be practically valuable strengths for surrogate modeling.

major comments (2)
  1. [Method and Experiments] The central composability claim—that lightweight Adaptation Blocks can assemble independently trained Foundation Blocks into an accurate surrogate for the full temporal evolution operator without significant interference—lacks supporting analysis or ablations for nonlinear operator interactions (e.g., convection-diffusion coupling or nonlinear advection in Burgers'). This is load-bearing for the reported relative L2 errors and robustness claims.
  2. [Abstract and Results] No numerical values for relative L2 errors, training details (epochs, learning rates, dataset sizes), statistical tests, or ablation studies on Adaptation Block design are provided, preventing assessment of whether the claimed improvements over PFNO/PDEFormer are meaningful or reproducible.
minor comments (1)
  1. [Method] Notation for the boundary-condition operator and its integration with the composed interior operator should be clarified with an explicit equation or diagram to confirm compatibility at inference.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive and detailed feedback. We address each major comment below and commit to revisions that strengthen the empirical support for our claims.

read point-by-point responses

  1. Referee: [Method and Experiments] The central composability claim—that lightweight Adaptation Blocks can assemble independently trained Foundation Blocks into an accurate surrogate for the full temporal evolution operator without significant interference—lacks supporting analysis or ablations for nonlinear operator interactions (e.g., convection-diffusion coupling or nonlinear advection in Burgers'). This is load-bearing for the reported relative L2 errors and robustness claims.

    Authors: We agree that explicit ablations are required to substantiate the composability claim under nonlinear interactions. In the revised manuscript we will add a dedicated ablation study that (i) measures the standalone error of each Foundation Block, (ii) quantifies the additional error introduced by the Adaptation Blocks on convection-diffusion and Burgers' flows, and (iii) compares compositional versus monolithic training on the same data. These results will be presented both numerically and via operator-norm visualizations to demonstrate that interference remains limited. revision: yes

  2. Referee: [Abstract and Results] No numerical values for relative L2 errors, training details (epochs, learning rates, dataset sizes), statistical tests, or ablation studies on Adaptation Block design are provided, preventing assessment of whether the claimed improvements over PFNO/PDEFormer are meaningful or reproducible.

    Authors: We acknowledge the absence of concrete numerical values and training specifications in the current version. The revised manuscript will include a new results table reporting mean relative L2 errors (plus standard deviations over five random seeds) for every PDE and baseline, together with the exact training protocol (200 epochs, learning rate 5e-4 with cosine annealing, 8000/2000 train/test splits per PDE). We will also add an ablation table on Adaptation Block depth and width, and paired t-test p-values confirming statistical significance of the reported gains. revision: yes

Circularity Check

0 steps flagged

No significant circularity; claims rest on external empirical validation

full rationale

The paper defines CompNO as a library of independently trained Foundation Blocks (parametric FNOs specialized to single operators) assembled by lightweight Adaptation Blocks, with a separate boundary operator. All performance claims (lower relative L2 error on linear systems, competitiveness on Burgers') are measured on the external PDEBench suite against named independent baselines (PFNO, PDEFormer). No equation or training step reduces a reported metric to a quantity defined by the fitted parameters themselves, and no uniqueness theorem or ansatz is smuggled via self-citation. The derivation chain is therefore self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 3 invented entities

The central claim rests on the domain assumption that PDE solution operators can be decomposed into independent fundamental operators that neural networks can learn separately and then recombine accurately.

axioms (1)
  • domain assumption Neural operators can learn and compose solution operators of parametric PDEs when decomposed into fundamental differential operators
    Core premise underlying the entire CompNO construction and assembly process.
invented entities (3)
  • Foundation Blocks no independent evidence
    purpose: Parametric Fourier neural operators each specialized to one fundamental operator such as convection or diffusion
    New modular components introduced to enable composition instead of monolithic training
  • Adaptation Blocks no independent evidence
    purpose: Lightweight modules that assemble Foundation Blocks into task-specific solvers
    New component required for the compositional framework
  • boundary-condition operator no independent evidence
    purpose: Dedicated operator that enforces Dirichlet boundary conditions exactly at inference time
    New mechanism claimed to achieve zero boundary loss

pith-pipeline@v0.9.0 · 5582 in / 1496 out tokens · 47019 ms · 2026-05-16T14:48:46.037630+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages · 4 internal anchors

  1. [1]

    Bayesian deep convolutional encoder–decoder networks for surro- gate modeling and uncertainty quantification,

    Y. Zhu and N. Zabaras, “Bayesian deep convolutional encoder–decoder networks for surro- gate modeling and uncertainty quantification,”Journal of Computational Physics, vol. 366, p. 415–447, 2018

    work page 2018

  2. [2]

    Prediction of aerody- namic flow fields using convolutional neural networks,

    S. Bhatnagar, Y. Afshar, S. Pan, K. Duraisamy, and S. Kaushik, “Prediction of aerody- namic flow fields using convolutional neural networks,”Computational Mechanics, vol. 64, no. 2, p. 525–545, 2019

    work page 2019

  3. [3]

    A deep fourier residual method for solving pdes using neural networks,

    J. M. Taylor, D. Pardo, and I. Muga, “A deep fourier residual method for solving pdes using neural networks,”Computer Methods in Applied Mechanics and Engineering, vol. 405, p. 115850, 2023

    work page 2023

  4. [4]

    Graph neural networks for mesh generation and adaptation in structural and fluid mechanics,

    U. Pelissier, A. Parret-Fr´ eaud, F. Bordeu, and Y. Mesri, “Graph neural networks for mesh generation and adaptation in structural and fluid mechanics,”Mathematics, vol. 12, no. 18, p. 2933, 2024

    work page 2024

  5. [5]

    Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,

    M. Raissi, P. Perdikaris, and G. Karniadakis, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,”Journal of Computational Physics, vol. 378, pp. 686–707, 2019

    work page 2019

  6. [6]

    Learning by neural networks under physical constraints for simu- lation in fluid mechanics,

    Y. Yang and Y. Mesri, “Learning by neural networks under physical constraints for simu- lation in fluid mechanics,”Computers & Fluids, vol. 248, p. 105632, 2022

    work page 2022

  7. [7]

    A hybrid physics-informed neural network for nonlinear partial differential equation,

    C. Lv, L. Wang, and C. Xie, “A hybrid physics-informed neural network for nonlinear partial differential equation,”International Journal of Modern Physics C, vol. 34, no. 06, p. 2350082, 2023

    work page 2023

  8. [8]

    Kolmogorov–arnold-informed neural network: A physics-informed deep learning frame- work for solving forward and inverse problems based on kolmogorov–arnold networks,

    Y. Wang, J. Sun, J. Bai, C. Anitescu, M. S. Eshaghi, X. Zhuang, T. Rabczuk, and Y. Liu, “Kolmogorov–arnold-informed neural network: A physics-informed deep learning frame- work for solving forward and inverse problems based on kolmogorov–arnold networks,” Computer Methods in Applied Mechanics and Engineering, vol. 433, p. 117518, 2025

    work page 2025

  9. [9]

    Neural operator: Learning maps between function spaces with applications to pdes,

    N. Kovachki, Z. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. Stuart, and A. Anand- kumar, “Neural operator: Learning maps between function spaces with applications to pdes,”Journal of Machine Learning Research, vol. 24, no. 89, pp. 1–97, 2023

    work page 2023

  10. [10]

    DeepONet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of operators

    L. Lu, P. Jin, and G. E. Karniadakis, “DeepONet: Learning nonlinear operators for iden- tifying differential equations based on the universal approximation theorem of operators,” Nature Machine Intelligence, vol. 3, no. 3, pp. 218–229, 2021. arXiv:1910.03193

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  11. [11]

    A deep learning framework for multi- operator learning: Architectures and approximation theory,

    A. Weihs, J. Sun, Z. Zhang, and H. Schaeffer, “A deep learning framework for multi- operator learning: Architectures and approximation theory,”ArXiv, vol. abs/2510.25379, 2025

    work page arXiv 2025

  12. [12]

    Fourier Neural Operator for Parametric Partial Differential Equations

    Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anand- kumar, “Fourier neural operator for parametric partial differential equations,”arXiv preprint arXiv:2010.08895, 2020

    work page internal anchor Pith review Pith/arXiv arXiv 2010

  13. [13]

    Geometry-informed neural operator for large-scale 3d pdes,

    Z. Li, N. Kovachki, C. Choy, B. Li, J. Kossaifi, S. Otta, M. A. Nabian, M. Stadler, C. Hundt, K. Azizzadenesheli,et al., “Geometry-informed neural operator for large-scale 3d pdes,”Advances in Neural Information Processing Systems, vol. 36, pp. 35836–35854, 2023. 20

    work page 2023

  14. [14]

    Learning the solution operator of parametric partial differential equations with physics-informed DeepONets,

    S. Wang, H. Wang, and P. Perdikaris, “Learning the solution operator of parametric partial differential equations with physics-informed DeepONets,”Science Advances, vol. 7, no. 40, p. eabi8605, 2021

    work page 2021

  15. [15]

    Parametric learning of time-advancement operators for unstable flame evolution,

    R. Yu and E. Hodzic, “Parametric learning of time-advancement operators for unstable flame evolution,”Physics of Fluids, vol. 36, no. 4, 2024

    work page 2024

  16. [16]

    LLaMA: Open and Efficient Foundation Language Models

    H. Touvron, T. Lavril, G. Izacard, X. Martinet, M.-A. Lachaux, T. Lacroix, B. Rozi` ere, N. Goyal, E. Hambro, F. Azhar,et al., “Llama: Open and efficient foundation language models,”arXiv preprint arXiv:2302.13971, 2023

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  17. [17]

    Hierarchical Text-Conditional Image Generation with CLIP Latents

    A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,”ArXiv, vol. abs/2204.06125, 2022

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  18. [18]

    High-resolution im- age synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High-resolution im- age synthesis with latent diffusion models,” in2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10674–10685, 2022

    work page 2022

  19. [19]

    Defining foun- dation models for computational science: A call for clarity and rigor,

    Y. Choi, S. W. Cheung, Y. Kim, P.-H. Tsai, A. N. Diaz, I. Zanardi, S. W. Chung, D. M. Copeland, C. Kendrick, W. Anderson, T. Iliescu, and M. Heinkenschloss, “Defining foun- dation models for computational science: A call for clarity and rigor,” 2025

    work page 2025

  20. [20]

    Fine-tune language models as multi-modal differential equation solvers,

    L. Yang, S. Liu, and S. J. Osher, “Fine-tune language models as multi-modal differential equation solvers,”Neural Networks, p. 107455, 2025

    work page 2025

  21. [21]

    PROSE: Predicting multiple operators and symbolic expressions using multimodal transformers,

    Y. Liu, Z. Zhang, and H. Schaeffer, “PROSE: Predicting multiple operators and symbolic expressions using multimodal transformers,”Neural Networks, vol. 180, p. 106707, 2024

    work page 2024

  22. [22]

    Multiple physics pretraining for spatiotemporal surrogate models,

    M. McCabe, B. R.-S. Blancard, L. H. Parker, R. Ohana, M. Cranmer, A. Bietti, M. Eick- enberg, S. Golkar, G. Krawezik, F. Lanusse, M. Pettee, T. Tesileanu, K. Cho, and S. Ho, “Multiple physics pretraining for spatiotemporal surrogate models,” inThe Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024

    work page 2024

  23. [23]

    Pretraining codomain attention neural operators for solving multiphysics pdes,

    M. A. Rahman, R. J. George, M. Elleithy, D. Leibovici, Z. Li, B. Bonev, C. White, J. Berner, R. A. Yeh, J. Kossaifi, K. Azizzadenesheli, and A. Anandkumar, “Pretraining codomain attention neural operators for solving multiphysics pdes,”Advances in Neural Information Processing Systems, vol. 37, 2024

    work page 2024

  24. [24]

    Bcat: A block causal transformer for pde foundation models for fluid dynamics.arXiv preprint arXiv:2501.18972, 2025

    Y. Liu, J. Sun, and H. Schaeffer, “BCAT: A block causal transformer for pde foundation models for fluid dynamics,”arXiv preprint arXiv:2501.18972, 2025

    work page arXiv 2025

  25. [25]

    PROSE-FD: A multimodal PDE foundation model for learning multiple operators for forecasting fluid dynamics, 2024

    Y. Liu, J. Sun, X. He, G. Pinney, Z. Zhang, and H. Schaeffer, “PROSE-FD: A multimodal pde foundation model for learning multiple operators for forecasting fluid dynamics,”arXiv preprint arXiv:2409.09811, 2024

    work page arXiv 2024

  26. [26]

    UPS: Efficiently building foundation models for PDE solving via cross-modal adaptation,

    J. Shen, T. Marwah, and A. Talwalkar, “UPS: Efficiently building foundation models for PDE solving via cross-modal adaptation,”Transactions on Machine Learning Research, 2024

    work page 2024

  27. [27]

    Poseidon: Efficient foundation models for PDEs,

    M. Herde, B. Raonic, T. Rohner, R. K¨ appeli, R. Molinaro, E. de Bezenac, and S. Mishra, “Poseidon: Efficient foundation models for PDEs,” inThe Thirty-eighth Annual Confer- ence on Neural Information Processing Systems, 2024

    work page 2024

  28. [28]

    Physix: A foundation model for physics simulations,

    T. Nguyen, A. Koneru, S. Li, and A. Grover, “Physix: A foundation model for physics simulations,” 2025. 21

    work page 2025

  29. [29]

    In-context operator learning with data prompts for differential equation problems,

    L. Yang, S. Liu, T. Meng, and S. J. Osher, “In-context operator learning with data prompts for differential equation problems,”Proceedings of the National Academy of Sci- ences, vol. 120, no. 39, 2023

    work page 2023

  30. [30]

    Vicon: Vision in-context operator networks for multi-physics fluid dynamics prediction.arXiv:2411.16063, 2024

    Y. Cao, Y. Liu, L. Yang, R. Yu, H. Schaeffer, and S. Osher, “Vicon: Vision in- context operator networks for multi-physics fluid dynamics prediction,”arXiv preprint arXiv:2411.16063, 2024

    work page arXiv 2024

  31. [31]

    PDEformer: Towards a foun- dation model for one-dimensional partial differential equations,

    Z. Ye, X. Huang, L. Chen, H. Liu, Z. Wang, and B. Dong, “PDEformer: Towards a foun- dation model for one-dimensional partial differential equations,” inICLR 2024 Workshop on AI4DifferentialEquations In Science, 2024

    work page 2024

  32. [32]

    Can we pre-train ICL-based SFMs for the zero-shot inference of the 1d CDR problem with noisy data?,

    M. Kang, D. Lee, W. Cho, K. Lee, A. Gruber, N. Trask, Y. Hong, and N. Park, “Can we pre-train ICL-based SFMs for the zero-shot inference of the 1d CDR problem with noisy data?,” inNeurips 2024 Workshop Foundation Models for Science: Progress, Opportuni- ties, and Challenges, 2024

    work page 2024

  33. [33]

    Guiding continuous operator learning through physics-based boundary constraints,

    N. Saad, G. Gupta, S. Alizadeh, and D. C. Maddix, “Guiding continuous operator learning through physics-based boundary constraints,” inThe Eleventh International Conference on Learning Representations, 2023

    work page 2023

  34. [34]

    Pdebench: An extensive benchmark for scientific machine learning,

    M. Takamoto, T. Praditia, R. Leiteritz, D. MacKinlay, F. Alesiani, D. Pfl¨ uger, and M. Niepert, “Pdebench: An extensive benchmark for scientific machine learning,”Ad- vances in Neural Information Processing Systems, vol. 35, pp. 1596–1611, 2022. 22

    work page 2022