REVIEW 3 major objections 6 minor 1 cited by
Mixture of neural operator experts for learning boundary conditions and model selection
T0 review · 3 major / 6 minor · reviewed 2026-08-08 · deepseek-v4-flash
Pith's one-line read A mixture of Fourier operator experts with partition-of-unity gating can learn non-periodic boundary conditions, select submodels, and produce an LES closure for turbulent channel flow.
desk verdict The POU-MOR-Physics combination is real and the disk result is clean, but the quarter-disk demonstration does not satisfy the advertised boundary condition; it deserves peer review with major revision. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the partition-of-unity mixture $P(u)(x)=\sum_{i} G_i(x)\, N_i(u)(x)$, where the $N_i$ are MOR-Physics operators acting in Fourier space and the $G_i$ are softmax-normalized gates that form a partition of unity and depend only on the spatial coordinate $x$. The load-bearing companion is the smooth extension of input functions to the periodic torus via the constrained problem $\min_{u_e} \int \nabla u_e \cdot \nabla u_e \, dx$ subject to $R u_e = u$, solved with a matrix-free conjugate gradient method; this extension is what prevents Gibbs phenomena and carries the boundary information into the Fourier representation.
What would settle it
Train POU-MOR-Physics on a domain with a reentrant corner, such as an L-shaped domain, and measure pointwise validation error against distance from the corner; if the error concentrates at the corner while the interior stays accurate, the $H^1$-extension premise is the limiting factor and the boundary-condition claim is only partially true.
Extended reading notes
Core claim
POU-MOR-Physics is a composite operator in which each expert is a MOR-Physics operator, a Fourier-domain composition of learned activations and modal weightings, and the experts are combined by softmax gates that depend only on the spatial coordinate. Inputs defined on a nontrivial domain are first extended to the periodic torus by minimizing the $H^1$ semi-norm subject to matching the data on the original domain, which suppresses Gibbs oscillations; the output is then restricted back. The mixture lets the model learn boundary behavior as a volumetric forcing, and one expert can be fixed to zero to encode no-slip walls. On the unit disk the validation $R^2$ exceeds 99.999%; on the quarter disk the learned solution operator achieves about 1% relative RMSE; and for Re=1000 channel flow a deterministic version reproduces filtered DNS fields with $R^2=98.8\%$, while the Bayesian version matches the energy spectrum and mean velocity statistics at ten flow-through times.
Load-bearing premise
The method's success rests on the smooth $H^1$-minimizing extension of the input to the periodic torus retaining enough boundary and interior information that the Fourier experts can learn the true restricted operator; if that extension destroys information, every learned operator built on it is compromised.
Editorial extensions
If this is right
- Non-periodic boundary conditions become a learnable feature instead of a hard constraint: the model discovers where to apply boundary forcing, as the zero expert on the disk walls demonstrates.
- The gating network produces an interpretable decomposition of the domain, with channel flow experiments showing separate experts for the boundary layer, the bulk, and the wall (zero) region.
- The same architecture can serve as a closure model for under-resolved simulations, since the learned LES operator is stable enough to be run autoregressively for ten flow-through times.
- Variational inference turns the operator into a probabilistic model whose predictive variance grows with time and flags out-of-distribution initial conditions, useful for deciding when a learned surrogate can be trusted.
Reading between the lines
- Because the gating depends only on position, the learned decomposition is static; conditioning the gates on the input field as well would let the model adapt its submodels to changing flow regimes, an extension the authors do not explore.
- The $H^1$-extension step is the most fragile part of the pipeline for domains with reentrant corners, where the minimizer itself develops a singularity; a testable extension would replace the $H^1$ objective with a weighted or boundary-adapted norm and compare errors near corners.
- The volume-penalization connection suggests a direct use beyond boundaries: the same mixture could discover coupled multi-physics systems, such as fluid-structure interaction, by learning separate experts for each physical regime without prescribing the interface.
- For LES, the uncertainty estimates could be turned into an online criterion for when the learned closure should yield to a coarse DNS solve, since the model already detects out-of-distribution initial fields from a single time slice.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces POU-MOR-Physics, a mixture-of-experts neural operator that combines Fourier-based MOR-Physics experts with a spatially conditioned partition-of-unity gating network. The method is designed to learn nonlinear operators on non-periodic domains by first smoothly extending input functions to the periodic torus and then letting spatially localized experts, including a zero expert, enforce boundary behavior and select among submodels. The authors demonstrate the approach on a synthetic nonlinear operator on the unit disk, a nonlinear Poisson problem with mixed boundary conditions on the quarter disk, and a large eddy simulation (LES) closure for Re=1000 turbulent channel flow using the JHTDB database, the last with mean-field variational inference to produce uncertainty estimates and out-of-distribution detection. The central claims are that the method recovers non-trivial boundary conditions, performs interpretable model selection, and yields LES predictions with calibrated-looking uncertainties far beyond the DNS training horizon.
Significance. If the results hold, the paper makes a useful conceptual contribution: the volume-penalization / mixture-of-experts analogy gives an interpretable way to impose boundary conditions in Fourier-based neural operators, and the learned zero expert is a clean mechanism for no-slip and Dirichlet behavior. The 2D disk example is well executed with very strong quantitative validation (R2>99.999%), and the visual expert partitions are compelling. The extension to LES with Bayesian variational inference, including an OOD detection test, addresses an important practical need. However, the paper does not release code or detailed training configurations, and, as detailed below, the quarter-disk boundary-condition demonstration and the LES validation are not yet sufficient to support the strength of the claims.
major comments (3)
- [Section 3.2, Eqs. (19)-(20)] The manufactured data for the quarter-disk Poisson problem do not satisfy the boundary conditions stated in Eq. (20). With ψ(x)=1−||x||²/2, on the unit circular arc ||x||=1 one has ψ=1/2, so v=(1/2)∑cos(f_{m,1}x1)cos(f_{m,2}x2)≠1, contradicting the advertised Dirichlet condition v=1 on that arc. In addition, Eq. (19) writes 'ˆu = ∇·tanh(∇u)', which mixes u and v; the intended relation is presumably u=∇·tanh(∇v), as in Eq. (20). As a result, the reported ~1% relative RMSE validates approximation of a manufactured mapping, not solution of the advertised mixed-boundary BVP. This is a load-bearing issue because the abstract and contributions explicitly cite recovery of non-trivial boundary conditions. The demonstration must be regenerated with data satisfying the stated BCs, or the claim about learning mixed boundary conditions must be revised.
- [Section 3.3, Table 1 and Figure 10] The LES evidence is too thin to support the central claim of 'accurately modeling Re=1000 3d wall bounded turbulence via neural operators and quantifying uncertainties.' The model is trained and evaluated on a single DNS trajectory, the deterministic R2=98.8% is reported without error bars or a specification of which fields and time steps it covers, and the spectral and statistical comparisons in Figure 10 are qualitative. The OOD experiment in Figure 11 is a single illustrative case. I recommend reporting ensemble or rollout statistics over multiple initial conditions, quantitative metrics with confidence intervals, and a calibration check for the MFVI predictive variance before the LES claims can be accepted.
- [Appendix C, Eq. (24)] As written, the LES update operator in Eq. (24) is not the Chorin projection it claims to be: the pressure-projection term has the opposite sign to the nonlinear term (so the updated field need not be divergence-free), the viscous term appears as -||κ||²/2 rather than νΔt||κ||², and no time step Δt or viscosity ν is defined. If these are notation or convention choices, please state them explicitly; otherwise the 'a priori known physics' component of the LES model is not correctly specified.
minor comments (6)
- [Section 3.2, Eq. (19)] The line 'ˆu = ∇·tanh(∇u)' uses u on both sides; it should presumably read u=∇·tanh(∇v), consistent with Eq. (20).
- [Section 3.3 and Figure 10] The text refers to 'Kologmorov’s 3/5’s rule' and the caption to 'Kolmogorov’s 5/3 rule'; the scaling exponent should be 5/3 consistently, and the typo 'Kologmorov' should be corrected.
- [Section 3.1] The paper reports validation R2>99.999% but does not state the train/test split, the Gaussian process kernel, the number of experts (I), the number of MOR layers (L), or the hyperparameters; adding a table of architecture and training details would aid reproducibility.
- [Section 3.3 and Table 1] Table 1 lists 'Time Dimension t=4000' but the text says the full time resolution is kept; please clarify whether the 4000 frames are all DNS output times or a subsample, and define the box filter and spatial stride precisely.
- [Section 2.2.2] The gating network is described as taking only coordinates as inputs, but Figure 8 shows a 3D partition; the paragraph should state the input dimension and architecture of the gating network for the LES case.
- [General] The manuscript does not include a data/code availability statement. Since the method is implemented with a complex distributed training setup, release of code or at least pseudocode and hyperparameters would be important for verification.
Circularity Check
No significant circularity: the operator, extension, and LES closure are defined by independent equations and fitted to external or synthetic benchmark data; the quarter-disk BC inconsistency is a validation flaw, not a self-referential derivation.
full rationale
The derivation chain is self-contained. The smooth extension in Section 2.1 (Eq. 4) minimizes the H1 semi-norm subject only to matching the input on X; it does not encode the target operator. The POU-MOR-Physics parameterization (Eq. 8) is a standard mixture-of-experts construction with spatially conditioned gates, and its parameters are fit to data by the independent losses in Eqs. (1), (2), and (18). The LES model (Eqs. 21-25) separates known Navier-Stokes/LES physics from the learned correction P, which is trained on filtered JHTDB DNS data, so the extrapolated posterior samples are outputs of a fitted operator rather than quantities defined by the claimed result. The paper relies on MOR-Physics and POU-Net references from overlapping authors (Patel and Desjardins 2018; Patel et al. 2021; Lee et al. 2021), but these supply component architectures and training tools; the central demonstration is carried by the paper's own equations and held-out evaluations, so the self-citations are not load-bearing. One serious flaw is flagged for correctness, not circularity: in Section 3.2, Eq. (19) sets psi(x)=1-||x||^2/2, so on the unit circular arc v=(1/2)*sum(cos(...)) != 1, contradicting the Dirichlet condition v=1 advertised in Eq. (20); the reported ~1% relative RMSE therefore validates approximation of a manufactured mapping, not solution of the stated mixed-boundary BVP. This undermines that demonstration, but it is an invalid benchmark, not a reduction of the model's prediction to its inputs.
Assumptions & free parameters
free parameters (4)
- Number of experts I =
3 (including zero expert)
- Number of MOR-Physics layers L
- Autoregressive training steps =
8
- LES data subsampling (spatial stride, box filter, time length) =
stride 20, box filter b=20, t=4000
assumptions (6)
- standard math Functions are represented on a periodic torus and manipulated via Fourier transforms.
- domain assumption A smooth H1-minimizing extension of each input function to the torus exists and is a faithful representation.
- ad hoc to paper The target operator can be written as a convex combination of a small number of experts with spatial-only gating.
- ad hoc to paper The LES closure and no-slip boundary effects can be represented as a learned operator applied pointwise after the Chorin-Euler update.
- domain assumption Box-filtered JHTDB DNS data is a sufficient and unbiased target for learning the filtered LES dynamics.
- domain assumption Mean-field Gaussian variational inference provides a useful posterior for uncertainty quantification.
Cite this review
Pith. "Pith review of Mixture of neural operator experts for learning boundary conditions and model selection." pith.science (2026). https://pith.science/paper/3ECJSJEC
@misc{pith2026250204562,
author = {Pith},
title = {Pith review of: Mixture of neural operator experts for learning boundary conditions and model selection},
year = {2026},
howpublished = {\url{https://pith.science/paper/3ECJSJEC}},
note = {Machine review of arXiv:2502.04562}
}
read the original abstract
While Fourier-based neural operators are best suited to learning mappings between functions on periodic domains, several works have introduced techniques for incorporating non trivial boundary conditions. However, all previously introduced methods have restrictions that limit their applicability. In this work, we introduce an alternative approach to imposing boundary conditions inspired by volume penalization from numerical methods and Mixture of Experts (MoE) from machine learning. By introducing competing experts, the approach additionally allows for model selection. To demonstrate the method, we combine a spatially conditioned MoE with the Fourier based, Modal Operator Regression for Physics (MOR-Physics) neural operator and recover a nonlinear operator on a disk and quarter disk. Next, we extract a large eddy simulation (LES) model from direct numerical simulation of channel flow and show the domain decomposition provided by our approach. Finally, we train our LES model with Bayesian variational inference and obtain posterior predictive samples of flow far past the DNS simulation time horizon.
Figures
Figures from the paper (14 more)
Forward citations
Cited by 1 Pith paper
-
SPAMoE: Spectrum-Aware Hybrid Operator Framework for Full-Waveform Inversion
SPAMoE reduces average MAE by 44.4% on ten OpenFWI sub-datasets via a spectral-preserving DINO encoder plus frequency-routed MoE of FNO, MNO and LNO experts.
Reference graph
Works this paper leans on
-
[1]
Stephen B. Pope. Turbulent Flows. Cambridge University Press, 2000
2000
-
[2]
Tianping Chen and Hong Chen. Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Transactions on Neural Networks, 6 0 (4): 0 911--917, 1995. doi:10.1109/72.392253
-
[3]
Nonlinear integro-differential operator regression with neural networks
Ravi G Patel and Olivier Desjardins. Nonlinear integro-differential operator regression with neural networks. arXiv preprint arXiv:1810.08552, 2018
arXiv 2018
-
[4]
Ravi G. Patel, Nathaniel A. Trask, Mitchell A. Wood, and Eric C. Cyr. A physics-informed operator regression framework for extracting data-driven continuum models. Computer Methods in Applied Mechanics and Engineering, 373: 0 113500, 2021. ISSN 0045-7825
work page 2021
-
[5]
Fourier neural operator for parametric partial differential equations
Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. In International Conference on Learning Representations, 2021
work page 2021
-
[6]
Learning nonlinear operators via deeponet based on the universal approximation theorem of operators
Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via deeponet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3 0 (3): 0 218--229, 2021. doi:10.1038/s42256-021-00302-5
-
[7]
U- NO : U-shaped neural operators
Md Ashiqur Rahman, Zachary E Ross, and Kamyar Azizzadenesheli. U- NO : U-shaped neural operators. Transactions on Machine Learning Research, 2023. ISSN 2835-8856
work page 2023
-
[8]
Tapas Tripura and Souvik Chakraborty. Wavelet neural operator for solving parametric partial differential equations in computational mechanics problems. Computer Methods in Applied Mechanics and Engineering, 404: 0 115783, 2023. ISSN 0045-7825. doi:https://doi.org/10.1016/j.cma.2022.115783
Show all 30 references
-
[9]
A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data
Lu Lu, Xuhui Meng, Shengze Cai, Zhiping Mao, Somdatta Goswami, Zhongqiang Zhang, and George Em Karniadakis. A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data. Computer Methods in Applied Mechanics and Engineering, 393: 0...
2022
-
[10]
Learning neural operators on riemannian manifolds
Chen, Gengxiang , Liu, Xu , Meng, Qinglu , Chen, Lu , Liu, Changqing , and Li, Yingguang . Learning neural operators on riemannian manifolds. Natl Sci Open, 3 0 (6): 0 20240001, 2024. doi:10.1360/nso/20240001. URL https://doi.org/10.1360/nso/20240001
2024
-
[11]
Fourier neural operator with learned deformations for pdes on general geometries
Zongyi Li, Daniel Zhengyu Huang, Burigede Liu, and Anima Anandkumar. Fourier neural operator with learned deformations for pdes on general geometries. J. Mach. Learn. Res., 24 0 (1), January 2023. ISSN 1532-4435
2023
-
[12]
Partition of unity networks: deep hp-approximation
Kookjin Lee, Nathaniel A Trask, Ravi G Patel, Mamikon A Gulian, and Eric C Cyr. Partition of unity networks: deep hp-approximation. arXiv preprint arXiv:2101.11256, 2021
2021 arXiv
-
[13]
Outrageously large neural networks: The sparsely-gated mixture-of-experts layer
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. arXiv preprint arXiv:1701.06538, 2017
2017 arXiv
-
[14]
A public turbulence database cluster and applications to study lagrangian evolution of velocity increments in turbulence
Yi Li, Eric Perlman, Minping Wan, Yunke Yang, Charles Meneveau, Randal Burns, Shiyi Chen, Alexander Szalay, and Gregory Eyink. A public turbulence database cluster and applications to study lagrangian evolution of velocity increments in turbulence. Journal of Turbulence, 0 (9)...
2008
-
[15]
A web services accessible database of turbulent channel flow and its use for testing a new integral wall model for les
J Graham, K Kanov, XIA Yang, M Lee, N Malaya, CC Lalescu, R Burns, G Eyink, A Szalay, RD Moser, et al. A web services accessible database of turbulent channel flow and its use for testing a new integral wall model for les. Journal of Turbulence, 17 0 (2): 0 181--215, 2016
2016
-
[16]
Data exploration of turbulence simulations using a database cluster
Eric Perlman, Randal Burns, Yi Li, and Charles Meneveau. Data exploration of turbulence simulations using a database cluster. In Proceedings of the 2007 ACM/IEEE Conference on Supercomputing, pages 1--11, 2007
2007
-
[17]
Variational inference: A review for statisticians
David M Blei, Alp Kucukelbir, and Jon D McAuliffe. Variational inference: A review for statisticians. Journal of the American statistical Association, 112 0 (518): 0 859--877, 2017
2017
-
[18]
Prediction of turbulent channel flow using fourier neural operator-based machine-learning strategy
Yunpeng Wang, Zhijie Li, Zelong Yuan, Wenhui Peng, Tianyuan Liu, and Jianchun Wang. Prediction of turbulent channel flow using fourier neural operator-based machine-learning strategy. Physical Review Fluids, 9 0 (8): 0 084604, 2024
2024
-
[19]
A characteristic based volume penalization method for general evolution problems applied to compressible viscous flows
Eric Brown-Dymkoski, Nurlybek Kasimov, and Oleg V Vasilyev. A characteristic based volume penalization method for general evolution problems applied to compressible viscous flows. Journal of Computational Physics, 262: 0 344--357, 2014
2014
-
[20]
A volume penalization method for incompressible flows and scalar advection--diffusion with moving obstacles
Benjamin Kadoch, Dmitry Kolomenskiy, Philippe Angot, and Kai Schneider. A volume penalization method for incompressible flows and scalar advection--diffusion with moving obstacles. Journal of Computational Physics, 231 0 (12): 0 4365--4383, 2012
2012
-
[21]
Numerical simulation of fluid--structure interaction with the volume penalization method
Thomas Engels, Dmitry Kolomenskiy, Kai Schneider, and J \"o rn Sesterhenn. Numerical simulation of fluid--structure interaction with the volume penalization method. Journal of Computational Physics, 281: 0 96--115, 2015
2015
-
[22]
A fourier spectral method for the navier--stokes equations with volume penalization for moving solid obstacles
Dmitry Kolomenskiy and Kai Schneider. A fourier spectral method for the navier--stokes equations with volume penalization for moving solid obstacles. Journal of Computational Physics, 228 0 (16): 0 5687--5709, 2009
2009
-
[23]
Constrained optimization and Lagrange multiplier methods
Dimitri P Bertsekas. Constrained optimization and Lagrange multiplier methods. Academic press, 2014
2014
-
[24]
Auto-encoding variational bayes
Diederik P Kingma. Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114, 2013
2013 arXiv
-
[25]
Weight uncertainty in neural network
Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural network. In International conference on machine learning, pages 1613--1622. PMLR, 2015
2015
-
[26]
What are bayesian neural network posteriors really like? In International conference on machine learning, pages 4629--4640
Pavel Izmailov, Sharad Vikram, Matthew D Hoffman, and Andrew Gordon Gordon Wilson. What are bayesian neural network posteriors really like? In International conference on machine learning, pages 4629--4640. PMLR, 2021
2021
-
[27]
Pytorch fsdp: experiences on scaling fully sharded data parallel
Yanli Zhao, Andrew Gu, Rohan Varma, Liang Luo, Chien-Chin Huang, Min Xu, Less Wright, Hamid Shojanazeri, Myle Ott, Sam Shleifer, et al. Pytorch fsdp: experiences on scaling fully sharded data parallel. arXiv preprint arXiv:2304.11277, 2023
2023 arXiv
-
[28]
Accurate, large minibatch sg d: training imagenet in 1 hour
P Goyal. Accurate, large minibatch sg d: training imagenet in 1 hour. arXiv preprint arXiv:1706.02677, 2017
2017 arXiv
-
[29]
Super-convergence: Very fast training of neural networks using large learning rates
Leslie N Smith and Nicholay Topin. Super-convergence: Very fast training of neural networks using large learning rates. In Artificial intelligence and machine learning for multi-domain operations applications, volume 11006, pages 369--386. SPIE, 2019
2019
-
[30]
Scheduled sampling for sequence prediction with recurrent neural networks
Samy Bengio, Oriol Vinyals, Navdeep Jaitly, and Noam Shazeer. Scheduled sampling for sequence prediction with recurrent neural networks. Advances in neural information processing systems, 28, 2015
2015
Reviewed August 8, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.