REVIEW 4 major objections 5 minor 7 references
A neural PDE solver built entirely from learned coordinate warps—no Fourier multipliers, no convolutions, no dot-product attention—matches or beats much larger baselines across flows, waves, and even a time-harmonic Helmholtz problem.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-02 22:45 UTC pith:GENFFDBN
load-bearing objection The real news is that a backbone built only from pointwise-predicted warps plus a U-Net scaffold beats FNO/conv/attention baselines across many The Well datasets — but the empirical case needs error bars and an ablation that isolates the warp primitive from the scaffold. the 4 major comments →
Flowers: A Warp Drive for Neural PDE Solvers
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim: multihead warps—learned coordinate pullbacks—are a sufficient and scalable nonlocal mixing primitive for PDE solution operators. A SELFWARP layer computes w(x) = V u(x + displacement(x)) per head, where the value map V and the displacement are predicted pointwise from u(x); the only spatial communication is sampling at displaced coordinates, one sparse sample per head. Stacked in residual blocks and a multiscale scaffold, this gives linear cost in grid size. Theory: conservation-law solutions are pullbacks along characteristics; wave solutions are weighted sums of pullbacks along rays (heads = ray directions); the continuum limit is a self-adaptive kinetic transport equati
What carries the argument
The paper's named primitive is the multihead SELFWARP layer: at each location x it predicts H displacement fields from a pointwise function of the local state u(x), then returns the concatenation of value features sampled at the displaced coordinates x + displacement(x). Its work is to inject adaptivity and nonlocality at linear cost, with pointwise displacement prediction and sparse sampling as the inductive bias. Around it, residual FlowerBlocks add channel mixing, normalization, and skip connections, and a multiscale encoder-decoder scaffold lets coarse scales move information across long ranges. Conceptually, the displacement field plays the role of the characteristic flow map in conserv
Load-bearing premise
The load-bearing premise is that a finite set of displacement fields predicted from only the local state, with one sparse source sample per head, can represent the necessary cross-space interactions in all benchmark regimes—including diffusion and time-harmonic Helmholtz—once embedded in multiscale residual blocks; the paper states this is not fully understood for those regimes.
What would settle it
On a linear heat-diffusion task, compare the full model against a variant with all displacements frozen to zero (pure pointwise mixing plus multiscale down/up-sampling). If the frozen variant matches, the warps are not the active mechanism. Alternatively, on the time-harmonic Helmholtz problem, compute the angular spectrum of each head's learned displacement field; unstructured spectra with high accuracy would contradict the ray-pullback explanation.
If this is right
- State-dependent sparse pullbacks plus pointwise channel mixing are enough for competitive PDE surrogates; Fourier multipliers, dense attention, and convolutional stencils are not required.
- With a fixed number of heads, cost stays linear in the number of grid points, making 3D and high-resolution problems tractable without sacrificing accuracy.
- The learned displacement fields are interpretable: without supervision they align with fluid velocity, organize into vortices, and track primary flow directions, suggesting the model discovers physical transport.
- Scaling is smooth: going from 17M to about 150M parameters steadily reduces error, and the largest variant improves over much larger, more expensive foundation models.
- The primitive transfers beyond transport: it performs well on diffusion-reaction systems and on a time-harmonic Helmholtz problem, a regime the authors expected to defeat it.
Where Pith is reading between the lines
- A practical consequence left implicit: if sparse warps suffice for transport-dominated physics, the field can redirect effort from designing global-mixing operators to designing head-sharing schemes and displacement regularizers.
- The displacement fields are a natural data source for identifying characteristic or ray structure; a testable extension is to compare head displacements on the Helmholtz problem with geometric-optics ray directions.
- The kinetic-theory lens predicts that dissipation is carried by the local mixing term rather than by the warps; shutting off local mixing while keeping warps should sharply degrade reaction-diffusion accuracy if this is right.
- Because displacement prediction is pointwise, the same primitive extends naturally to point clouds and neighbor graphs, which would give a geometry-agnostic backbone for unstructured meshes.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces FLOWERS, a neural PDE surrogate architecture whose core primitive is a multihead warp layer: each head predicts a pointwise displacement field from the local input, and nonlocality is introduced only by sampling features at the displaced coordinates. The layer is embedded in residual blocks and a U-Net-like multiscale scaffold with pointwise channel mixing. Three theoretical lenses are offered as motivation: characteristics of conservation laws, ray/pullback structure of waves, and a kinetic continuum limit. Empirically, the paper reports results on The Well, PDEGym, PDEBench, and a WaveBench Helmholtz problem, claiming that a 17M-parameter FLOWERS model outperforms FNO, CNextU-Net, and SCOT baselines on next-step and autoregressive rollout tasks, and that scaling to about 150M parameters is competitive with POSEIDON-L on one Euler dataset. Ablations and data-efficiency experiments are also included, and the authors explicitly acknowledge several limitations, including that the Helmholtz result is an 'expected failure case' that is not yet understood.
Significance. If the empirical results are robust, the paper proposes a genuinely simple and efficient nonlocal mixing primitive — pointwise-predicted multihead warps — that could serve as an alternative to Fourier multipliers, attention, and convolutional mixing in neural PDE solvers. The paper is unusually honest in framing the theoretical sections as motivational rather than predictive, and it ships code, a broad benchmark suite, ablation studies, data-efficiency experiments, and a scaling study. The main contributions are the architectural proposal and the breadth of evidence. However, the central claim that the warp primitive itself is responsible for the observed performance is not cleanly isolated from the U-Net scaffold, and the statistical reporting is too weak to fully support the 'consistently outperforms' language.
major comments (4)
- [§4.1, Tables 1, 2, 4–9] All quantitative comparisons are single runs. Table 5 shows that per-model and per-dataset learning rates were selected from three values on validation, yet no seed variation, standard deviations, or confidence intervals are reported anywhere. Consequently, the claims that FLOWERS 'consistently outperforms' and is 'best by a substantial margin' are not statistically supported. Please report means and standard deviations over at least three seeds, and either fix the learning-rate protocol or report sensitivity to it.
- [§2.1, App. C.4, Tables 3 and 9] The central architectural claim — that pointwise multihead warps are a sufficient nonlocal primitive — is not isolated. In the main models, SELFWARP is embedded in a U-Net whose DownConv and UpConv layers are learned stride-2 and transposed convolutions that aggregate spatial information across scales. Table 9 shows that removing the scaffold substantially worsens errors (e.g., active matter 0.0567 vs. 0.0249; Rayleigh–Bénard 0.1565 vs. 0.0807; TRL 2D 0.2658 vs. 0.1930; viscoelastic 0.0882 vs. 0.0624). Table 3 shows that warping is essential, but it does not compare the same scaffold with an alternative adaptive nonlocal mixer (e.g., attention or Fourier layers) in place of learned warps. The benchmark wins therefore do not demonstrate that the warp mechanism, rather than the scaffold's convolutional aggregation, provides the global interactions. Please add same-scaffold ablations replac
- [§3.2, App. E, §5.1] For wave and Helmholtz problems, the theoretical motivation gives a pullback representation where the source coordinate Γ_{-t}(x, η) is determined by the ray structure of the global speed field, not by a pointwise function of the local state u(x). The quadrature in Eqs. (2) and (24) has no controlled error bound, and the paper explicitly labels the Helmholtz benchmark an 'expected failure case' that surprisingly works, adding that the reason is not fully understood. Given that the abstract claims adaptive, global interactions from pointwise-predicted displacements, this is a load-bearing gap in the current evidence. Please provide either a controlled experiment on a wave problem with known rays (e.g., checking whether learned displacements align with Γ_{-t}) or a finite-H error analysis, or soften the claim accordingly.
- [§4.2, Table 2] The abstract's scaling claim — that a 150M-parameter variant 'improves over recent transformer-based foundation models with much more parameters, data, and training compute' — rests on a single dataset (Euler multi quadrants) and a single baseline (POSEIDON-L). This is too narrow to support a general foundation-model comparison. Please either add more datasets and baselines for the scaling study or restrict the claim to the specific dataset used.
minor comments (5)
- [Throughout] Model names are inconsistent: 'SCOT' and 'scOT' are used interchangeably, and 'CNUNET' appears alongside 'CNEXTU-NET' in Tables 1 and 9. Please unify.
- [Abstract and §1] The phrase 'no convolutional mixing' is easy to misread, since the U-Net scaffold in App. C.4 uses stride-2 and transposed convolutions. Please clarify that the claim concerns in-scale feature mixing, or explicitly state that the scaffold uses convolutions only for resolution changes.
- [App. A.3] The duplicate-frame correction in the viscoelastic dataset and the bug in The Well benchmark code are important for reproducibility. Please provide more detail (e.g., code version/commit, which datasets are affected, and the magnitude of the effect on the reported numbers).
- [Table 4 and Table 8] It is unclear whether the learning rates for the Helmholtz and conditioned 1→1 experiments were tuned or fixed. Please state the protocol for these tables.
- [Figures 6 and 13] The displacement-field visualizations would be easier to interpret with a short caption explaining arrow scaling, head indexing, and how the field is spatially subsampled for display.
Circularity Check
No circularity: the theoretical vignettes are explicitly motivational and the empirical claims are grounded in external benchmarks; self-citations are not load-bearing.
full rationale
The paper separates two claims: (1) SELFWARP/FLOWERS is an effective PDE-surrogate primitive, supported by controlled comparisons against baselines on The Well, PDEBench, PDEGym, and WaveBench; and (2) three theoretical vignettes connect the architecture to characteristics, rays, and a kinetic limit. I checked whether (2) is used to manufacture (1) by fitting, definition, or self-citation. It is not. The theory sections are explicitly non-causal: 'These perspectives are primarily motivational: we do not claim that any one of them causally explains the architecture’s empirical success' (Section 3), and Section 5.1 calls the diffusive/Helmholtz arguments 'heuristic'. The conservation-law derivation (Appendix B) derives a first-order pointwise displacement x - t A(u0(x)) and then observes that SELFWARP implements the same local structure; the wave parametrix (Appendix E) produces a pullback representation and notes the analogy to multihead warps. These are structural correspondences, not predictions obtained from fitted constants. The kinetic limit (Appendix D) is a formal continuum limit: it restates the layer operation as a kinetic PDE under scaling assumptions, but the paper does not use that PDE to predict benchmark outcomes. Self-citations (Kothari et al. 2020; Benitez et al. 2025; WaveBench) appear as prior motivation, related kinetic schemes, or a benchmark source; no load-bearing step reduces to a self-citation. The weakest assumption—that pointwise-predicted displacements with one sparse source sample per head suffice across regimes—is explicitly flagged as heuristic, including calling Helmholtz an 'expected failure case' (Appendix A.3). That is an unsupported assumption or correctness risk, not circularity. No specific circular step can be quoted, so the score is 0.
Axiom & Free-Parameter Ledger
free parameters (4)
- head count H =
40 (tiny), 80 (small), 120 (medium)
- lift channels c0 =
160 (tiny), 320 (small), 480 (medium)
- U-Net levels L =
4
- per-model learning rate =
varies by dataset and model (Table 5)
axioms (5)
- standard math Before shock formation, scalar conservation law solutions are constant along characteristics and the short-time inverse flow map is a pointwise-dependent displacement (Appendix B, Eqs. 11-13).
- domain assumption In the absence of caustics, high-frequency wave solutions are superpositions of pullbacks along rays (Appendix E, Eq. 1/22).
- ad hoc to paper Cascaded FlowerBlocks have a kinetic continuum limit as forward Euler in layer index, with a self-adaptive streaming velocity and collision operator (Appendix D, Eq. 17).
- domain assumption Bilinear interpolation on a structured grid with periodic/zero-padding boundary extension is a faithful surrogate for the continuous pullback.
- domain assumption The Well benchmark data and splits are valid once the authors' duplicate-frame removal and benchmark-code bug correction are applied.
invented entities (1)
-
Kinetic phase-space variable eta indexing heads
no independent evidence
read the original abstract
We introduce Flowers, a neural architecture for learning PDE solution operators built entirely from multihead warps. Aside from pointwise channel mixing and a multiscale scaffold, Flowers use no Fourier multipliers, no dot-product attention, and no convolutional mixing. Each head predicts a displacement field and warps the mixed input features. Motivated by physics and computational efficiency, displacements are predicted pointwise, without any spatial aggregation, and nonlocality enters only through sparse sampling at source coordinates, one per head. Stacking warps in multiscale residual blocks yields Flowers, which implement adaptive, global interactions at linear cost. We theoretically motivate this design through three complementary lenses: flow maps for conservation laws, waves in inhomogeneous media, and a kinetic-theoretic continuum limit. Flowers achieve excellent performance on a broad suite of 2D and 3D time-dependent PDE benchmarks, particularly flows and waves. A compact 17M-parameter model consistently outperforms Fourier, convolution, and attention-based baselines of similar size, while a 150M-parameter variant improves over recent transformer-based foundation models with much more parameters, data, and training compute.
Figures
Reference graph
Works this paper leans on
-
[5]
Pant, P., Doshi, R., Bahl, P., and Barati Farimani, A
URL https://openreview.net/forum? id=00Sx577BT3. Pant, P., Doshi, R., Bahl, P., and Barati Farimani, A. Deep learning for reduced order modelling and efficient tem- poral evolution of fluid simulations.Physics of Flu- ids, 33(10):107101, 10 2021. ISSN 1070-6631. doi: 10.1063/5.0062546. URL https://doi.org/10. 1063/5.0062546. Ronneberger, O., Fischer, P., ...
-
[18]
URL https://www.sciencedirect.com/ science/article/pii/S0021999118302341. 13 Flowers: A Warp Drive for Neural PDE Solvers FNO CNEXTU-NET SCOT FLOWER Optimal LR1×10 −3 1×10 −3 5×10 −4 1×10 −3 VRMSE 0.0987 0.1425 0.30680.0463 Table 4.Predictions on the 15Hz acoustic time-harmonic Helmholtz equation problem from WaveBench. Map of Appendices • Appendix A cont...
arXiv 2019
-
[2021]
Fielding, D
URL https://openreview.net/forum? id=YicbFdNTTy. Fielding, D. B., Ostriker, E. C., Bryan, G. L., and Jermyn, A. S. Multiphase gas and the fractal nature of radiative tur- bulent mixing layers.The Astrophysical Journal Letters, 894(2):L24, 2020. Franco, N. R., Manzoni, A., Zunino, P., and Hesthaven, J. S. Deep orthogonal decomposition: a continuously adapt...
2020
-
[2022]
Hao, Z., Wang, Z., Su, H., Ying, C., Dong, Y ., Liu, S., Cheng, Z., Song, J., and Zhu, J
URL https://openreview.net/forum? id=EXHG-A3jlM. Hao, Z., Wang, Z., Su, H., Ying, C., Dong, Y ., Liu, S., Cheng, Z., Song, J., and Zhu, J. GNOT: A general neu- ral operator transformer for operator learning. InInter- national Conference on Machine Learning, pp. 12556– 12569. PMLR, 2023. Hao, Z., Su, C., Liu, S., Berner, J., Ying, C., Su, H., Anand- kumar,...
Pith/arXiv arXiv 2023
-
[2023]
cc/paper_files/paper/2023/file/ e15790966a4a9d85d688635c88ee6d8a-Paper-Conference
URL https://proceedings.neurips. cc/paper_files/paper/2023/file/ e15790966a4a9d85d688635c88ee6d8a-Paper-Conference. pdf. Takamoto, M., Praditia, T., Leiteritz, R., MacKinlay, D., Alesiani, F., Pfl ¨uger, D., and Niepert, M. Pdebench: An extensive benchmark for scientific machine learning. Advances in neural information processing systems, 35: 1596–1611, 2...
2023
-
[2024]
McCabe, M., Mukhopadhyay, P., Marwah, T., Blancard, B
URL https://openreview.net/forum? id=DKSI3bULiZ. McCabe, M., Mukhopadhyay, P., Marwah, T., Blancard, B. R.-S., Rozet, F., Diaconu, C., Meyer, L., Wong, K. W. K., Sotoudeh, H., Bietti, A., Espejo, I., Fear, R., Golkar, S., Hehir, T., Hirashima, K., Krawezik, G., Lanusse, F., Morel, R., Ohana, R., Parker, L., Pettee, M., Shen, J., Cho, K., Cranmer, M., and ...
Pith/arXiv arXiv 2025
-
[8856]
Liu, T., Benitez, J
URL https://openreview.net/forum? id=EPPqt3uERT. Liu, T., Benitez, J. A. L., Faucher, F., Khorashadizadeh, A., de Hoop, M. V ., and Dokmani ´c, I. WaveBench: Benchmarking data-driven solvers for linear wave propa- gation PDEs.Transactions on Machine Learning Re- search, 2024. ISSN 2835-8856. URL https:// openreview.net/forum?id=6wpInwnzs8. 11 Flowers: A W...
2024
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.