Pith. sign in

REVIEW 3 major objections 4 minor 49 references

PatternFormer: Learning Multiple Solution Patterns in Reaction--Diffusion Systems

T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read PatternFormer recovers the complete set of coexisting steady states of nonlinear reaction–diffusion PDEs in a single generative pass, and reports Turing patterns absent from the reference data.

desk verdict PatternFormer is a genuine step forward in learning solution sets for multistable PDEs, but its completeness claims inherit the completeness of the reference solvers. read the letter →

arxiv 2608.12286 v1 pith:WJMXKWHC submitted 2026-08-12 math-ph math.MP

Zhipeng Chang , Wenpeng Yin , Wenrui Hao This is my paper · ORCID
classification math-phmath.MP MSC 35B3235K5768T07
keywords multiplesolutionsset-valuedoperatorreaction-diffusionsystemsTuringpatternsautoregressivegenerationlargelanguagemodelsphysicsresiduallosssolutionlandscape
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PatternFormer is a method for learning set-valued solution operators for nonlinear PDEs: instead of mapping a parameter to one solution, it maps it to the whole set $S(p)$ of coexisting steady states. The framework turns the unordered set into a canonical sequence ordered by a signed integral $\Phi(u)$ and generates that sequence autoregressively with a frozen pretrained language model. On nonlinear elliptic problems where the multiplicity $K(p)$ is known it emits every branch in one forward pass; on the Gray–Scott system it produces coexisting Turing patterns, including physically valid states absent from the reference data. The paper argues this replaces branch-by-branch numerical search with a single generative prediction that lands inside the attraction basin of each branch, and that sequential fine-tuning across equations is a step toward foundation models for solution landscapes.

What carries the argument

The central object is the set-valued operator $G: p \mapsto \{u_1,\ldots,u_{K(p)}\}$ realized as autoregressive sequence generation in the continuous embedding space of a frozen pretrained large language model. A frozen autoencoder compresses each field into a latent vector $z \in \mathbb{R}^{256}$, a matching layer projects $[z;p]$ into the model embedding, low-rank adaptation fine-tunes the attention projections, and a dual head predicts the next latent plus a stop marker. The canonical ordering by the signed integral $\Phi(u)=\int_0^1 u\,dx$ is what converts an unordered solution set into a sequence that causal attention can generate, and the hinged residual penalty $\mathrm{ReLU}(R(\hat{u}_k;p)-\epsilon)$ is what drives the free-run tail of Gray–Scott outputs toward previously unseen valid steady states.

What would settle it

Take a parameter inside the training range, run an independent certified continuation or deflation solver to count the nontrivial branches, and compare with PatternFormer's single deterministic pass: if the certified count exceeds what the model emits, or if any emitted field fails to refine to a field satisfying the PDE residual, the set-completeness claim is refuted.

Watch

Extended reading notes

Core claim

The central claim is that the map $p \mapsto S(p)$, where $S(p)$ is the complete set of coexisting steady-state solutions, can be learned directly by an autoregressive transformer operating on latent encodings of solution fields. Coexisting states are unordered, so the method imposes a canonical ordering by a scalar functional (the signed integral or the spatial mean of the activator field) and serializes the set into a sequence; a classification head stops generation when the multiplicity $K(p)$ is known, while a fixed-budget design with a hinged physics-residual loss handles open-ended families. On three nonlinear elliptic benchmark problems the generated set matches the reference branches and refines to solver precision, and on Gray–Scott the model discovers 63 distinct coexisting patterns at one parameter, 61 of which were not in the reference data. The paper concludes that direct generation performs the essential discovery, with Newton refinement only improving precision.

Load-bearing premise

The load-bearing premise is that the numerical solvers used to produce and evaluate the reference solution sets are complete: for the elliptic examples the companion or homotopy solver must miss no branch, and for Gray–Scott the reference set is only a lower bound, so claims about genuinely new patterns rest on an acknowledged incomplete enumeration.

Editorial extensions

If this is right

  • For parametrized PDE families with known multiplicity, a single deterministic pass recovers the complete branch set; Newton iteration from those outputs converges in about three to four iterations and reduces median relative $L^2$ error to roughly $10^{-8}$.
  • The same frozen encoder transfers across grid resolutions from $N=1024$ to $N=4096$ without retraining, and PF-seeded refinement recovers the full solution set one to two orders of magnitude faster than a multiscale numerical solver.
  • Seeding continuation from the predicted set transports the full branch structure far outside the training range: Example 1 reaches $p=100$ (training range $p\le 18$), and Example 2 keeps 100% branch coverage across the sampled $(a_2,a_4)$ plane.
  • On Gray–Scott, noise-perturbed autoregressive passes accumulate distinct valid patterns; at one representative parameter the model finds 63 distinct patterns, 61 of them absent from the reference data, and across test parameters the average rises to 8.39 discoveries per parameter.
  • Warm-starting from an earlier PatternFormer checkpoint beats fresh pretrained initialization on a new equation, so the learned representation transfers across multistable systems rather than being retrained from scratch.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The completeness claim is inherited: if the reference solvers miss a branch, PatternFormer cannot learn it, so the strongest test of the method would be evaluation against an independent certified branch count rather than the same solvers used to build the training sets.
  • The canonical-ordering-plus-autoregressive design is not specific to PDEs: any finite set of valid outputs indexed by a scalar ordering (molecular conformations, mechanical equilibria, optical modes) could be generated the same way.
  • The Gray–Scott 'new' patterns are accepted on a fixed finite-difference residual below $10^{-9}$; checking them under mesh refinement and linear stability analysis would confirm they are true steady states and not discretization artifacts.
  • Sequential warm-starting across Gray–Scott diffusion regimes suggests the accumulated representation could eventually serve as a general solution-landscape prior, which is the implicit trajectory the paper sketches toward scientific foundation models.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes PatternFormer (PF), an autoregressive transformer framework that learns a set-valued solution operator G: p -> S(p) for multistable reaction-diffusion and elliptic PDEs. PF uses a frozen pretrained LLM (Qwen2.5-7B) with LoRA adaptation in a continuous latent space, canonical ordering of coexisting solutions by a scalar functional, a termination head for problems with known multiplicity, and a fixed-budget generation mode with physics-residual regularization for open-ended discovery. The authors demonstrate the approach on three nonlinear elliptic benchmark problems where PF is claimed to recover all solution branches in a single forward pass, and on the Gray-Scott system where PF is claimed to discover diverse coexisting Turing patterns, including states absent from the reference data and at out-of-distribution parameters. Additional experiments study extrapolation, warm-starting across equations, and the separation between direct generation and numerical refinement.

Significance. If the central claims are substantiated, this is a valuable methodological contribution: it addresses a real limitation of single-valued neural operators by proposing a concrete architecture for learning solution sets, and it provides reproducible code, fixed seeds, and regenerable data. The warm-starting experiments and the explicit attempt to separate discovery from refinement (Fig. 5) are commendable. The main weakness is that the completeness of the solution sets for the elliptic examples is inherited from the classical solvers that generated the reference data, and the Gray-Scott 'new patterns' claim rests on the same residual criterion used by the refinement solver. These issues do not invalidate the method, but they require the claims to be qualified and the evidence to be strengthened.

major comments (3)
  1. [Results: Learning complete solution landscapes; Methods: Ground-truth data generation] The central claim that PF recovers the complete solution set S(p) for Examples 1-3 is conditional on the completeness of the reference sets produced by the companion-based multilevel FEM and homotopy continuation. The termination head is trained on exactly those solution counts, so a branch missed by the reference solver is invisible to both training and evaluation. The manuscript's stated motivation for the stop-token design is that 'the number of coexisting solutions can be determined theoretically,' but for Examples 1 and 3 the counts come from the numerical solver, not from an independent theoretical enumeration. Please state this dependence explicitly, report results as 'complete with respect to the reference solver,' and add an independent completeness check (e.g., deflated continuation or interval-verified counting) on at least a subset of test parameters.
  2. [Results: Fig. 2B; Abstract] The abstract says PF 'recovers all solution branches in one inference step' for the nonlinear elliptic problems, but Fig. 2B contains the annotation 'missing branch p ~ 16.1 16.2, 17.8 17.9, 0.8% of test params.' As printed, this is a direct counterexample to the unqualified completeness claim. Even if the 99.9% figure refers to Newton convergence and the 0.8% refers to missing branches, the text does not reconcile these numbers. Please report complete-set recovery and per-branch recall separately, and revise the abstract and Results to match the measured rates.
  3. [Results: Discovering coexisting Turing patterns; Methods: Inference and post-processing] For Gray-Scott, a pattern is accepted when the finite-difference residual is below 1e-9 after refinement by the same quasi-Newton tensor-product solver used to build the reference set, and the reference set itself is acknowledged to be only a lower bound. Therefore 'physically valid states absent from the reference data' should be read as 'discrete steady states of the discretized equations not contained in the lower-bound dataset,' not as independent physical validation. In addition, because every generated candidate is Newton-refined, the relative contribution of the generator versus the refinement solver is not quantified for Gray-Scott; the 'Separating solution discovery from numerical refinement' section addresses only Examples 1-3. I ask for pre-refinement residual statistics for Gray-Scott outputs and independent verification of a sample of new patterns using a different discretization or solver.
minor comments (4)
  1. [Abstract and Introduction] There are missing spaces in 'developPatternFormer' and 'introducePatternFormer'; these should be corrected.
  2. [Figure 2D] The wall-clock comparison should specify the hardware used for the LLM inference and for the traditional solver, and whether model loading and data-transfer times are included, so that the reported speedups are reproducible.
  3. [Methods: Inference and post-processing] The deduplication threshold is stated as relative L2 below 0.15; the sensitivity of the reported solution counts to this threshold should be reported, especially for the Gray-Scott '63 distinct patterns' claim.
  4. [Methods: Sequence construction and training] The hinged residual threshold epsilon is described only as 'chosen between' residuals of valid and non-solution fields; the main text should give the actual value or a precise selection criterion, or explicitly defer to Supplementary Section 5 for all three problems.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular step meets the quote-and-reduction standard; the claims are supervised holdout predictions and residual-validated discoveries, with solver completeness as an external-validity caveat rather than a circularity.

full rationale

PatternFormer's central map G:p->S(p) is learned from reference solution sets produced by classical numerical solvers (companion-based multilevel FEM for Examples 1 and 3, homotopy continuation for Example 2, quasi-Newton tensor-product solver for Gray-Scott). These reference sets are inputs, not outputs of the model, and the evaluation is performed on held-out parameters, so the reported branch recovery, multiplicity accuracy, and coverage are genuine predictions that could in principle fail. No fitted parameter is later renamed as a prediction, and no equation is defined in terms of the claimed result. The self-citations to [7], [9], [42] are citations to reproducible numerical methods rather than to an unverified uniqueness theorem invoked to force the model's choice; the paper explicitly acknowledges that for Gray-Scott the reference set is only a lower bound. The Gray-Scott discovery claim is validated by requiring the discretized equations to have residual below 1e-9 after Newton refinement; this residual check is related to the physics regularizer in Eq. 6, but the novelty relative to the reference data is an independent empirical comparison, and the generated states are not selected by construction from the training labels. The main weakness is that the completeness of the elliptic reference sets is inherited from the classical solvers, which is an external-validity/correctness caveat rather than a circular derivation. Under the hard-rule standard requiring an explicit reduction Eq. X = Eq. Y or a fitted parameter renamed as a prediction, no circular step is present.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The central claim rests on a pipeline of numerical solvers and on several hand-chosen thresholds. The most consequential free parameter is the residual hinge threshold epsilon, which decides whether a tail output is penalized; the completeness of the reference data is an assumption that is not independently verified. No new physical entities are postulated.

free parameters (5)
  • Hinge residual threshold epsilon = not reported in main text
    Chosen between the residual of valid steady states and the residual of non-solution fields. This threshold controls which tail outputs are penalized and therefore what counts as a discovered solution.
  • Fixed Gray-Scott budget K = 24
    Number of solution slots in the fixed-budget design. All Gray-Scott results depend on this capacity choice.
  • Deduplication threshold = relative L2 = 0.15
    Fields are considered distinct when pairwise relative L2 exceeds 0.15; changing this threshold changes the reported counts of discovered patterns.
  • Noise level sigma for latent perturbations = 0.05, 0.1, 0.2 in Figure 3D
    Controls exploration in the open-ended Gray-Scott experiments; discovery counts grow with sigma and with the number of forward passes.
  • Loss weights lambda_mse, lambda_lat, lambda_pde, lambda_ce = not reported in main text
    Weighted reconstruction and physics losses. These are hand-chosen and not derived from first principles.
assumptions (5)
  • domain assumption Governing PDEs (Eq. 1 and Eq. 2) are correct models of the target systems.
    The entire training and validation set is generated from these equations, and physical validity is defined by their residuals.
  • domain assumption Reference solvers enumerate the complete set of steady-state solutions for the elliptic examples.
    Training labels and termination counts come from companion-based multilevel FEM or homotopy continuation; completeness is not independently certified.
  • domain assumption Canonical ordering by Phi (signed integral or mean activator) is a stable, deterministic ordering of coexisting solutions.
    The model relies on this ordering to serialize unordered solution sets into sequences; ties are resolved by field energy.
  • domain assumption Finite-difference residual R approximates the PDE residual well enough for the hinge loss to guide outputs to genuine steady states.
    The residual is computed on a fixed discretization and is used both as the training loss and as the validation criterion.
  • domain assumption Frozen Qwen2.5-7B pretrained weights provide a beneficial inductive bias for continuous scientific sequence generation.
    The paper's pretraining comparison supports this empirically, but it is not a theorem and is specific to this architecture.

how reviews work

0 comments
Cite this review

Pith. "Pith review of PatternFormer: Learning Multiple Solution Patterns in Reaction--Diffusion Systems." pith.science (2026). https://pith.science/paper/WJMXKWHC

@misc{pith2026260812286,
  author       = {Pith},
  title        = {Pith review of: PatternFormer: Learning Multiple Solution Patterns in Reaction--Diffusion Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WJMXKWHC}},
  note         = {Machine review of arXiv:2608.12286}
}
read the original abstract

Many nonlinear models across physics, chemistry, and biology exhibit multiple solutions for the same parameters, and capturing this entire solution set is essential for understanding pattern-forming systems. Yet existing learned surrogates are fundamentally single-valued: neural operators map each parameter to a single output, and physics-informed neural networks converge to one branch. We develop \textbf{PatternFormer} (PF), a large language model-based framework for learning the multiple solutions of nonlinear partial differential equations. By transforming unordered coexisting solutions into canonical sequences, PF produces structured solution sets in a single autoregressive pass, terminating automatically for finite families and enforcing physical residual constraints for unbounded ones. On nonlinear elliptic problems it recovers all solution branches in one inference step; on Gray--Scott it generates coexisting Turing patterns, including physically valid states absent from the reference data and beyond training. PF can also be sequentially fine-tuned across multistable systems, toward general foundation models for solution landscapes.

Figures

Figures reproduced from arXiv: 2608.12286 by the authors.

Figure 1
Figure 1. PatternFormer as a set-valued solution operator. A, Conventional neural operators learn a single-valued mapping from a parameter p to one solution u(p), whereas PatternFormer (PF) learns a set-valued mapping from p to the complete set S(p) of coexisting solutions. B, Context construction: each in-context example consists of a parameter and its complete solution set, with coexisting states ordered by solution norm an… view at source ↗
Figure 2
Figure 2. Learning complete solution landscapes of nonlinear elliptic PDEs. A, Representative predictions of PF for Example 1 at five parameter values p with increasing solution multiplicity k. Direct model outputs (dashed) are compared with reference solutions (solid). B, Relative L 2 errors before and after refinement, number of Newton iter￾ations, and final residual ∥F∥, grouped by solution multiplicity k. C, Recovered bif… view at source ↗
Figure 3
Figure 3. Open-ended discovery of coexisting Turing patterns in the Gray–Scott system. Statistics are computed over held-out test parameters at DA=6.25 × 10−5 and DS=1.25 × 10−4 . A, Pattern multiplicity landscape over the (ρ, µ) parameter plane. Each cell shows the largest number of distinct steady patterns discovered by PF; orange cells indicate out-of-distribution parameter regions, whereas grey cells contain only the homo… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Extrapolation beyond the training parameter range. A,B, Recovery of reference solution branches as parameters move beyond the training domain for Example 1 (A, training region p ≤ 18) and Example 3 (B, training region s ≤ 1600). Direct PF generation at the extrapolated…
Figure 5
Figure 5. Figure 5: PatternFormer discovers solution branches; refinement improves numerical precision. A,B, For Ex￾ample 1 (A) and Example 2 (B), direct PF predictions (orange dashed) and Newton-refined solutions (blue dotted) compared with reference solution branches (black solid). The …
Figure 6
Figure 6. Figure 6: Pretraining and cross-equation warm-starting improve recovery of coexisting solutions. A–D, Ex￾ample 1: deterministic PatternFormer fine-tuned from pretrained Qwen2.5-7B compared with the same architecture initialized from random weights. A, Validation MSE during the t…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

49 extracted references · 42 canonical work pages

  1. [1]

    M. C. Cross and P. C. Hohenberg. Pattern formation outside of equilibrium.Reviews of Modern Physics, 65(3): 851–1112, 1993

    work page 1993

  2. [2]

    Gray and S

    P. Gray and S. K. Scott. Autocatalytic reactions in the isothermal, continuous stirred tank reactor: oscillations and instabilities in the systemA+ 2B→3B,B→C.Chemical Engineering Science, 39(6):1087–1097, 1984

    work page 1984

  3. [3]

    John E. Pearson. Complex patterns in a simple system.Science, 261(5118):189–192, 1993

    work page 1993

  4. [4]

    Alan M. Turing. The chemical basis of morphogenesis.Philosophical Transactions of the Royal Society of London B, 237(641):37–72, 1952

    work page 1952

  5. [5]

    Gierer and H

    A. Gierer and H. Meinhardt. A theory of biological pattern formation.Kybernetik, 12(1):30–39, 1972

    1972

  6. [6]

    Solution landscape of reaction-diffusion systems reveals a nonlinear mechanism and spatial robustness of pattern formation.Fundamental Research, 2025

    Shuonan Wu, Bing Yu, Yuhai Tu, and Lei Zhang. Solution landscape of reaction-diffusion systems reveals a nonlinear mechanism and spatial robustness of pattern formation.Fundamental Research, 2025

    work page 2025

  7. [7]

    Spatial pattern formation in reaction–diffusion models: a computational approach

    Wenrui Hao and Chuan Xue. Spatial pattern formation in reaction–diffusion models: a computational approach. Journal of Mathematical Biology, 80(2):521–543, 2020

    work page 2020

  8. [8]

    Matrix-oriented discretization methods for reaction–diffusion PDEs: comparisons and applications.Computers and Mathematics with Applications, 79: 2067–2085, 2020

    Maria Chiara D’Autilia, Ivonne Sgura, and Valeria Simoncini. Matrix-oriented discretization methods for reaction–diffusion PDEs: comparisons and applications.Computers and Mathematics with Applications, 79: 2067–2085, 2020

    work page 2020

Show all 49 references
  1. [9]

    Wenrui Hao, Sun Lee, and Young Ju Lee. Companion-based multi-level finite element method for computing multiple solutions of nonlinear differential equations.Computers and Mathematics with Applications, 168:162– 173, 2024

    2024

  2. [10]

    A homotopy method based on weno schemes for solving steady state problems of hyperbolic conservation laws

    Wenrui Hao, Jonathan D Hauenstein, Chi-Wang Shu, Andrew J Sommese, Zhiliang Xu, and Yong-Tao Zhang. A homotopy method based on weno schemes for solving steady state problems of hyperbolic conservation laws. Journal of Computational Physics, 250:332–346, 2013

    2013

  3. [11]

    A bootstrapping approach for computing multiple solutions of differential equations.Journal of Computational and Applied Mathematics, 258:181–190, 2014

    Wenrui Hao, Jonathan D Hauenstein, Bei Hu, and Andrew J Sommese. A bootstrapping approach for computing multiple solutions of differential equations.Journal of Computational and Applied Mathematics, 258:181–190, 2014

    2014

  4. [12]

    P. E. Farrell, Á. Birkisson, and S. W. Funke. Deflation techniques for finding distinct solutions of nonlinear partial differential equations.SIAM Journal on Scientific Computing, 37(4):A2026–A2045, 2015

    2015

  5. [13]

    E. G. Charalampidis, P. G. Kevrekidis, and P. E. Farrell. Computing stationary solutions of the two-dimensional Gross–Pitaevskii equation with deflated continuation.Communications in Nonlinear Science and Numerical Simulation, 54:482–499, 2018. doi: 10.1016/j.cnsns.2017.05.024

    2018 doi

  6. [14]

    Wei Zhu, Hong-Kun Zhang, and P. G. Kevrekidis. Machine learning of independent conservation laws through neural deflation.Physical Review E, 108(2):L022301, 2023

    2023

  7. [15]

    L. Lu, P. Jin, G. Pang, Z. Zhang, and G. E. Karniadakis. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators.Nature Machine Intelligence, 3:218–229, 2021

    2021

  8. [16]

    Z. Li, N. Kovachki, K. Azizzadenesheli, B. Liu, K. Bhattacharya, A. Stuart, and A. Anandkumar. Fourier neural operator for parametric partial differential equations.International Conference on Learning Representations (ICLR), 2021

    2021

  9. [17]

    Raissi, P

    M. Raissi, P. Perdikaris, and G. E. Karniadakis. Physics-informed neural networks: a deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations.Journal of Compu- tational Physics, 378:686–707, 2019

    2019

  10. [18]

    Giampaolo, M

    F. Giampaolo, M. De Rosa, P. Qi, S. Izzo, and S. Cuomo. Physics-informed neural networks approach for 1D and 2D Gray-Scott systems.Advanced Modeling and Simulation in Engineering Sciences, 9:5, 2022

    2022

  11. [19]

    Zongren Zou, Zhicheng Wang, and George Em Karniadakis. Learning and discovering multiple solutions using physics-informed neural networks with random initialization and deep ensemble.Proceedings of the Royal Society A, 481(2325):20250205, 2025

    2025

  12. [20]

    When and why PINNs fail to train: a neural tangent kernel perspective.Journal of Computational Physics, 449:110768, 2022

    Sifan Wang, Xinling Yu, and Paris Perdikaris. When and why PINNs fail to train: a neural tangent kernel perspective.Journal of Computational Physics, 449:110768, 2022

    2022

  13. [21]

    Stiff-PINN: physics-informed neural network for stiff chemical kinetics.The Journal of Physical Chemistry A, 125(36):8098–8106, 2021

    Weiqi Ji, Weilun Qiu, Zhiyu Shi, Shaowu Pan, and Sili Deng. Stiff-PINN: physics-informed neural network for stiff chemical kinetics.The Journal of Physical Chemistry A, 125(36):8098–8106, 2021

    2021

  14. [22]

    Haoyang Zheng, Yao Huang, Ziyang Huang, Wenrui Hao, and Guang Lin. HomPINNs: homotopy physics- informed neural networks for solving the inverse problems of nonlinear differential equations with multiple solutions.Journal of Computational Physics, 500:112751, 2024. 16

    2024

  15. [23]

    Geometry-informed neural networks.Proceedings of the International Conference on Machine Learning (ICML), 2025

    Arturs Berzins, Andreas Radler, Eric V olkmann, Sebastian Sanokowski, Sepp Hochreiter, and Johannes Brand- stetter. Geometry-informed neural networks.Proceedings of the International Conference on Machine Learning (ICML), 2025

    2025

  16. [24]

    Newton informed neural operator for computing multiple solutions of nonlinear partial differential equations.Advances in Neural Information Processing Systems (NeurIPS), 2024

    Wenrui Hao, Xinliang Liu, and Yahong Yang. Newton informed neural operator for computing multiple solutions of nonlinear partial differential equations.Advances in Neural Information Processing Systems (NeurIPS), 2024

    2024

  17. [25]

    A neural network kernel decomposition for learning multiple steady states in parameterized dynamical systems.Preprint at arXiv:2312.10315, 2024

    Yimeng Zhang, Alexander Cloninger, Bo Li, and Xiaochuan Tian. A neural network kernel decomposition for learning multiple steady states in parameterized dynamical systems.Preprint at arXiv:2312.10315, 2024

    2024 arXiv

  18. [26]

    Wang and W

    Y . Wang and W. Hao. Learning the evolution operator for nonlinear PDEs.Preprint at arXiv:2502.05571, 2025

    2025

  19. [27]

    C. Rao, P. Ren, Q. Wang, O. Buyukozturk, H. Sun, and Y . Liu. Encoding physics to learn reaction–diffusion processes.Nature Machine Intelligence, 5:765–779, 2023

    2023

  20. [28]

    Katsiaryna Haitsiukevich, Onur Poyraz, Pekka Marttinen, and Alexander Ilin. Diffusion models as probabilis- tic neural operators for recovering unobserved states of dynamical systems.IEEE International Workshop on Machine Learning for Signal Processing (MLSP), 2024

    2024

  21. [29]

    Andersson, Andrew El-Kadi, Dominic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, Remi Lam, and Matthew Willson

    Ilan Price, Alvaro Sanchez-Gonzalez, Ferran Alet, Tom R. Andersson, Andrew El-Kadi, Dominic Masters, Timo Ewalds, Jacklynn Stott, Shakir Mohamed, Peter Battaglia, Remi Lam, and Matthew Willson. Probabilistic weather forecasting with machine learning.Nature, 637:84–90, 2025

    2025

  22. [30]

    Pawan Kumar, Emilien Dupont, Francisco J

    Bernardino Romera-Paredes, Mohammadamin Barekatain, Alexander Novikov, Matej Balog, M. Pawan Kumar, Emilien Dupont, Francisco J. R. Ruiz, Jordan S. Ellenberg, Pengming Wang, Omar Fawzi, Pushmeet Kohli, and Alhussein Fawzi. Mathematical discoveries from program search with larg...

    2024

  23. [31]

    L. Yang, S. Liu, T. Meng, and S. J. Osher. In-context operator learning with data prompts for differential equation problems.Proceedings of the National Academy of Sciences, 120(39):e2310142120, 2023

    2023

  24. [32]

    L. Yang, T. Meng, S. Liu, and S. J. Osher. Prompting in-context operator learning with sensor data, equations, and natural language.Preprint at arXiv:2308.05061, 2023

    2023 arXiv

  25. [33]

    In-context learning of linear systems: generalization theory and applications to operator learning.Preprint at arXiv:2409.12293, 2024

    Frank Cole, Yulong Lu, Wuzhe Xu, and Tianhao Zhang. In-context learning of linear systems: generalization theory and applications to operator learning.Preprint at arXiv:2409.12293, 2024

    2024 arXiv

  26. [34]

    Unsupervised solution operator learning for mean-field games via sampling- invariant parametrizations.Journal of Computational Physics, 2025

    Han Huang and Rongjie Lai. Unsupervised solution operator learning for mean-field games via sampling- invariant parametrizations.Journal of Computational Physics, 2025

    2025

  27. [35]

    Wang, Pierre Erbacher, and Patrick Gallinari

    Louis Serrano, Armand Kassaï Koupaï, Thomas X. Wang, Pierre Erbacher, and Patrick Gallinari. Zebra: in- context generative pretraining for solving parametric PDEs.Proceedings of the International Conference on Machine Learning (ICML), 2025

    2025

  28. [36]

    ENMA: tokenwise autoregression for generative neural PDE operators.Preprint at arXiv:2506.06158, 2025

    Armand Kassaï Koupaï, Lise Le Boudec, Louis Serrano, and Patrick Gallinari. ENMA: tokenwise autoregression for generative neural PDE operators.Preprint at arXiv:2506.06158, 2025

    2025

  29. [37]

    Pretrained transformers as universal computation engines.AAAI Conference on Artificial Intelligence, 2022

    Kevin Lu, Aditya Grover, Pieter Abbeel, and Igor Mordatch. Pretrained transformers as universal computation engines.AAAI Conference on Artificial Intelligence, 2022

    2022

  30. [38]

    Efficient approximation of deep ReLU networks for functions on low dimensional manifolds

    Minshuo Chen, Haoming Jiang, Wenjing Liao, and Tuo Zhao. Efficient approximation of deep ReLU networks for functions on low dimensional manifolds. InAdvances in Neural Information Processing Systems (NeurIPS), volume 32, 2019

    2019

  31. [39]

    Towards a foundation model for partial differential equations: Multi-operator learning and extrapolation.Physical Review E, 111(3):035304, 2025

    Jingmin Sun, Yuxuan Liu, Zecheng Zhang, and Hayden Schaeffer. Towards a foundation model for partial differential equations: Multi-operator learning and extrapolation.Physical Review E, 111(3):035304, 2025

    2025

  32. [40]

    E. J. Hu, Y . Shen, P. Wallis, Z. Allen-Zhu, Y . Li, S. Wang, L. Wang, and W. Chen. LoRA: low-rank adaptation of large language models.International Conference on Learning Representations (ICLR), 2022

    2022

  33. [41]

    Qwen2.5 technical report.Preprint at arXiv:2412.15115, 2025

    An Yang et al. Qwen2.5 technical report.Preprint at arXiv:2412.15115, 2025

    2025 arXiv

  34. [42]

    An efficient quasi-Newton method with tensor product imple- mentation for solving quasi-linear elliptic equations and systems.Journal of Scientific Computing, 103:89, 2025

    Wenrui Hao, Sun Lee, and Xiangxiong Zhang. An efficient quasi-Newton method with tensor product imple- mentation for solving quasi-linear elliptic equations and systems.Journal of Scientific Computing, 103:89, 2025

    2025

  35. [43]

    Stepien, Erica M

    Tracy L. Stepien, Erica M. Rutter, and Yang Kuang. A data-motivated density-dependent diffusion model of in vitro glioblastoma growth.Mathematical Biosciences and Engineering, 12(6):1157–1172, 2015

    2015

  36. [44]

    Aronow, and Leili Shahri- yari

    Arkadz Kirshtein, Shaya Akbarinejad, Wenrui Hao, Trang Le, Sumeyye Su, Rachel A. Aronow, and Leili Shahri- yari. Data driven mathematical model of colon cancer progression.Journal of Clinical Medicine, 9(12):3947, 2020

    2020

  37. [45]

    Rempala, and Hye-Won Kang

    Jae Kyoung Kim, Grzegorz A. Rempala, and Hye-Won Kang. Reduction for stochastic biochemical reaction networks with multiscale conservations.Multiscale Modeling & Simulation, 15(4):1376–1403, 2017. 17

    2017

  38. [46]

    A topology-based network tree for the prediction of protein– protein binding affinity changes following mutation.Nature Machine Intelligence, 2(2):116–123, 2020

    Menglun Wang, Zixuan Cang, and Guo-Wei Wei. A topology-based network tree for the prediction of protein– protein binding affinity changes following mutation.Nature Machine Intelligence, 2(2):116–123, 2020

    2020

  39. [47]

    Order matters: sequence to sequence for sets.International Conference on Learning Representations (ICLR), 2016

    Oriol Vinyals, Samy Bengio, and Manjunath Kudlur. Order matters: sequence to sequence for sets.International Conference on Learning Representations (ICLR), 2016

    2016

  40. [48]

    End-to-end object detection with transformers.European Conference on Computer Vision (ECCV), 2020

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers.European Conference on Computer Vision (ECCV), 2020

    2020

  41. [49]

    Fleet, and Geoffrey Hinton

    Ting Chen, Saurabh Saxena, Lala Li, David J. Fleet, and Geoffrey Hinton. Pix2seq: a language modeling framework for object detection.International Conference on Learning Representations (ICLR), 2022. Acknowledgments Z.C. and W.H. were supported by National Institute of General...

    2022

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.