REVIEW 3 major objections 6 minor 3 references
Reformulating Neural Operators in $d+1$ Dimensions for Embedding Evolution
T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Neural operators that evolve embeddings along an auxiliary Fourier dimension achieve lower PDE errors with training cost comparable to FNO.
desk verdict A novel d+1 operator architecture with a broad empirical sweep, but the controlled evidence for its central mechanism is confounded and the results lack error bars and code. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the auxiliary dimension $p$ and the warped-phase lift $v(x,p)=w^T(p)a(x)$, which turns a $d$-dimensional function evolution into a $(d+1)$-dimensional phase-space evolution. Along $p$, the model learns an operator of the form $F_p^{-1}\tilde{A}F_p+b$---a Fourier-diagonal differential operator plus bias that discretizes a general linear PDE along $p$ with an Euler step---so that embedding evolution is learned as an operator rather than as a dense matrix. Combined with truncated spectral convolutions and a local differential operator on $x$, this operator-based evolution is what the paper argues gives SKNO its accuracy and resolution stability.
What would settle it
Train SKNO and FNO with matched compute on a benchmark where the auxiliary-dimension coupling is known to be densely nonlocal, e.g., a manufactured operator whose kernel couples all $p$ values with comparable strength rather than acting as a local differential operator. If SKNO's advantage over FNO disappears or reverses, the linear-PDE form in Eq. (13) is the operative limitation.
Extended reading notes
Core claim
The paper proposes to bring embedding evolution into the operator formalism by redefining neural operators in $d+1$ dimensions: lift the input $a(x)$ to $v(x,p)=w(p)^T a(x)$, evolve $v$ with kernel integral operators over $D_x \times D_p$, and recover the output by $u(x)=\int_{D_p}\chi(p)v(x,p)\,dp$. Along the auxiliary dimension, the evolution is parameterized as a learnable Fourier-domain linear PDE, $F_p^{-1}\tilde{A}F_p+b$, discretized with an Euler step. The resulting Schr\"odingerised Kernel Neural Operator (SKNO) is claimed to achieve the lowest relative $L_2$ error among the evaluated baselines on a spectrum of benchmark PDEs, while remaining stable under mixed-resolution training, super-resolution inference, and zero-shot generalization to unseen time horizons.
Load-bearing premise
The paper assumes that the evolution of the lifted signal along the auxiliary dimension is governed by a linear PDE of the form Eq. (13), and that a single learnable Fourier-domain operator plus bias with a fixed Euler step captures it faithfully; if a problem's embedding dynamics resist that representation, the theoretical motivation and the claimed advantage of SKNO would not follow.
Editorial extensions
If this is right
- SKNO offers a concrete alternative to embedding widening: instead of increasing embedding width to raise capacity, the model adds an auxiliary dimension governed by a learnable linear PDE, which the paper argues is cheaper while improving accuracy.
- The $d+1$ formulation makes the auxiliary dimension a first-class part of the operator, aligning the architecture with Schr\"odingerisation; the paper credits this alignment for SKNO's ability to capture sharp features such as material interfaces and vorticity.
- Resolution-invariance experiments show SKNO degrades less than FNO when tested on grids far beyond the training resolution, directly supporting applications like super-resolution forecasting from a single trained model.
- Ablations attribute a substantial part of the gains to the Fourier-domain operator and bias along $p$; removing either raises error, indicating the linear-PDE inductive bias is load-bearing for the reported performance.
- Zero-shot temporal generalization on 2D Navier-Stokes means the operator learned on seen time steps remains accurate on unseen future steps, improving autoregressive rollout usability.
Reading between the lines
- Because the $p$-axis is learned as a linear operator, a natural testable extension is to run SKNO with adaptively chosen grid sizes along $p$ at inference: if the learned operator is genuinely resolution-invariant along $p$, the same weights should give accurate predictions on denser $p$-grids than those used in training, without retraining.
- The sparsity and energy-capture results suggest the auxiliary dimension behaves like a learned dictionary; one could test whether SKNO's $p$-space can be pruned, e.g., with a sparse $\tilde{A}$, to shrink model size while preserving accuracy---an extension not explored in the paper.
- If the linear-PDE assumption holds for a given problem family, the spectrum of the learned operator $\tilde{A}$ may be interpretable as an effective equation discovered for the embedding; comparing these spectra across benchmark families could reveal common structural features across PDEs.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a d+1-dimensional reformulation of neural operators, in which the usual d-dimensional embedding evolution is lifted to a (d+1)-dimensional domain by introducing an auxiliary dimension p, motivated by the Schrödingerisation method for PDEs. The resulting architecture, the Schrödingerised Kernel Neural Operator (SKNO), uses Fourier-based operators in both the physical domain and the auxiliary dimension, together with residual connections and a final local differential propagator. The authors claim consistent state-of-the-art relative L2 errors across more than ten benchmarks (1D heat, advection, Burgers; 2D Darcy, Gray-Scott, shallow-water, Navier-Stokes, stress/strain; 3D compressible Navier-Stokes and Rayleigh-Taylor; ERA5 wind), with training time comparable to FNO, and additionally report resolution-invariance, super-resolution, and zero-shot temporal generalization experiments, as well as ablations of the lifting/recovering operators and of individual architectural components.
Significance. If the claims hold, the auxiliary-dimension operator design provides a simple and computationally efficient alternative to brute-force embedding scaling in neural operators, and the d+1 perspective could be a useful organizing principle for future operator architectures. The empirical scope is broad and the ablation study (Table 3) is informative. The manuscript also makes a concrete theoretical claim about efficient energy capture (Theorem D.3) and proposes an entanglement-entropy diagnostic. However, the conclusions are currently weakened by the absence of repeated-seed statistics, by a controlled linear-block comparison that does not isolate the p-evolution mechanism, and by an unproved inclusion at the heart of the main theoretical result. The core idea is plausible and the full-model results are suggestive, but the evidence as presented is not yet at the level required to establish the central mechanistic and SOTA claims.
major comments (3)
- [Sec. 3.1, Eq. (15); Table 1; Table 7 (1D Heat and 1D Advection rows)] The Table 1 comparison is confounded and therefore does not provide the claimed controlled evidence for the auxiliary-dimension evolution. With L=1, SKNO has zero global spectral propagators and consists only of the final local propagator, which is the untruncated physical-space differential kernel of Eq. (15), in addition to the W_l residual/p-evolution block. The FNO (1 mode) and Transolver (1-4 slices) columns are band-limited in x. Consequently, the ~340x gap on heat and the analogous advection gap could arise entirely from the untruncated local differential branch, which is absent from the FNO and Transolver blocks. To support the statement in Sec. 4.2 that SKNO "removes this bottleneck" along p, please report the heat and advection errors for (i) SKNO with the F_p^{-1} A F_p + b module ablated (keeping the local differential branch), and (ii) SKNO and baselines with L>=2 so that all models include a global spectral/attention branch. Without these controls, Table 1 cannot be cited as evidence for the d+1 mechanism.
- [Appendix D, Theorem D.3, paragraph following Eq. (41)] The key inclusion H_FNO ⊂ H_SKNO is asserted without proof or construction. The proof of Eq. (42) and the entire claim that SKNO captures energy more efficiently than FNO depend on this inclusion, yet no argument is given for why every FNO linear map is realizable by SKNO under the same width budget, nor why the supremum over H_SKNO dominates. Since the paper presents Theorem D.3 as a formal proof, please either provide an explicit parameter assignment (including how the p-evolution module and local propagator are set to reproduce an arbitrary FNO layer) or explicitly label the statement as a conjecture and support it empirically. The sentence "It is straightforward to extend the above arguments to function spaces" is not a substitute for this missing justification.
- [Sec. 4.1-4.2, Tables 2, 7, 13, 15] The central SOTA claim is not statistically supported. All tables report single runs without error bars, seeds, or code, and several margins over the best baseline are very small (e.g., Burgers 5.475e-4 vs 6.479e-4 in Table 2; shallow-water 0.0027 vs 0.0030 in Table 13; stress 0.0472 vs 0.0481 in Table 17). In addition, the main comparisons are not strictly budget-matched: SKNO uses 4+1 layers (four global propagators plus one local propagator) on Darcy, Navier-Stokes, and Rayleigh-Taylor, whereas FNO uses 4 layers (Table 7), and the local propagator adds an untruncated module. Please report mean and standard deviation over at least three seeds, and release the code and data so that the reported single-run numbers can be independently verified.
minor comments (6)
- [Title] The title contains a spacing error: "ind+ 1Dimensions" should read "in d+1 Dimensions".
- [Sec. 2.1, Eq. (6)] In the definition of D_mu, the index range "mu_{-Np/2},...,mu_{Np/2}" is incomplete; it should specify the ordering of the DFT frequencies, e.g., "mu_{-Np/2},...,mu_{Np/2-1}" for an even grid.
- [Fig. 1 caption] The phrase "a group of Schrödinger / convection equations" is grammatically awkward; consider rephrasing to "a family of uncoupled Schrödinger equations (after Fourier transform in p) and a family of uncoupled convection equations (after Fourier transform in x)".
- [Sec. 4.2 and Appendix B, paragraph on spectral decay] The sentence "As shown in Fig. 5 (Li et al., 2021) which illustrates the spectral decay of truncation modes" appears to cite the reference for its own figure; if Fig. 5 is the paper's own figure, the citation should be removed or moved to the text.
- [Appendix D, Theorem D.4, Eqs. (46)-(53)] Several equations have unmatched parentheses (e.g., "Grad(bV^(SKNO)_{L-1})_{ij} =" in Eq. (47) is missing a closing bracket). Please carefully proofread the display math.
- [Appendix B, "Averaging Neural Operator on Dp"] The reference to "ANO (Lanthaler et al., 2025)" seems inconsistent with the content of the cited paper (which is about universality of nonlinear operators); please verify that the reference supports the averaging-operator claim.
Circularity Check
No significant circularity: SKNO's benchmark results are empirical and self-contained; the d+1 Schrödingerisation framing is motivation, not a fitted prediction.
full rationale
The central claims of the paper are empirical relative-L2 comparisons (Tabs 1, 2, 13, 15) of models trained end-to-end on independent train/test splits under the relative-L2 loss (Eqs. 16–17). No quantity in the Schrödingerisation derivation (Example 1, Eqs. 1–6) is fitted to test data and then reported as a prediction: lifting w(p), recovering χ(p), Fourier weights Ã, and bias b are ordinary learned network parameters, and the d+1 framework (Eqs. 7–10) defines the architecture family rather than supplying its fitted values. The only overlapping-author citation (Li et al., 2025, cited in the introduction for the context that NOs can outperform common NNs) is not load-bearing for SKNO's design or for any benchmark result; the load-bearing theory citations are external (Jin et al. for Schrödingerisation; Kovachki et al. 2021 for universal approximation). Appendix Theorem D.3's inequality follows from the definitional set inclusion H_FNO⊂H_SKNO ('Since H_FNO⊂H_SKNO, the supremum over H_SKNO cannot be smaller...'), which is a best-case superset statement, not an empirical prediction and not the source of the reported SOTA numbers. The Table 1 comparison may be confounded by SKNO's extra local differential operator, but a baseline confound is an external-validity/correctness risk, not a circular reduction: the error gaps are not equal by construction to the fitted parameters. Under the quoted-evidence standard, no circular step is present.
Assumptions & free parameters
free parameters (3)
- Auxiliary grid size Np =
24 to 32 in main experiments; 4 to 16 in linear-block comparisons
- Number of Fourier modes (truncation) =
1 to 16 depending on benchmark
- Step size Δt =
1
assumptions (4)
- standard math Universal approximation theorem for neural operators (Kovachki et al., 2021) applies to the d+1 dimensional spectral construction
- domain assumption The lifting v0(x,p) = w(p)^T a(x) is a sufficiently expressive preparation of the d+1 dimensional signal
- ad hoc to paper The evolution of the embedding along p is governed by a linear PDE of the form Eq. 13 and is faithfully implemented by F_p^{-1} \tilde{A} F_p + b
- ad hoc to paper H_FNO ⊂ H_SKNO (SKNO can realize every FNO linear map under the same width budget)
invented entities (2)
-
Auxiliary dimension p
-
Schrödingerised Kernel Neural Operator (SKNO)
Cite this review
Pith. "Pith review of Reformulating Neural Operators in $d+1$ Dimensions for Embedding Evolution." pith.science (2026). https://pith.science/paper/QF56AU25
@misc{pith2026250511766,
author = {Pith},
title = {Pith review of: Reformulating Neural Operators in $d+1$ Dimensions for Embedding Evolution},
year = {2026},
howpublished = {\url{https://pith.science/paper/QF56AU25}},
note = {Machine review of arXiv:2505.11766}
}
abstract
Neural Operators (NOs) are powerful architectures for learning mappings between function spaces. While most advances focus on refining kernel parameterizations over the $d$-dimensional physical domain, the evolution of lifted embeddings remains underexplored, which often drives models toward computationally expensive embedding-scaling designs to improve approximation. In this paper, we introduce an auxiliary function dimension that models embedding evolution in operator form, thereby reformulating the NO pipeline in $d+1$ dimensions. We instantiate this framework via Fourier-based operators acting jointly on the physical and auxiliary domains, yielding a basis-diversified auxiliary evolution module as an alternative to brute-force embedding scaling. Across more than ten increasingly challenging benchmarks, ranging from the 1D heat equation to the highly nonlinear 3D Rayleigh-Taylor instability, our model consistently achieves the lowest relative $L_2$ error among the evaluated baselines. Crucially, this advantage is empirically supported by (1) controlled budget-aware comparisons against scaled and ablated baselines; (2) robustness under mixed-resolution training and super-resolution inference; and (3) zero-shot generalization to unseen temporal regimes. In addition, we present a broader set of design choices for lifting and recovery operators, demonstrating their impact on our model's predictive performance.
Figures
Figures from the paper (13 more)
Reference graph
Works this paper leans on
-
[1]
Approximating an operatorM is equivalent (via the Fourier-conjugate decomposition) to approximating the conjugate mapping(s){ cMi}kx i=1 (and, after also Fourier-transforming inp, the conjugates{ fcMi})
-
[2]
The forward DFT/frequency-projection and the inverse discrete transform admit neural operator approximation on the spectral domain (See Lemma 7 and Lemma 8 in (Kovachki et al., 2021), hence the operator learning problem reduces to a finite-dimensional neural-network approximation on the compact coefficient sets ˆVi, ˜ˆVi. In particular, treating the auxil...
-
[2021]
which illustrates the spectral decay of truncation modes, our chosen benchmarks 6 and 7 contain significantly more high-frequency modes than simpler benchmarks such as 3 or other incompressible NS benchmarks withν >1×10 −4. To further evaluate the capability of these efficient and lightweight neural operators in capturing complex non-linear dynamics in 3D...
work page 2021
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.