ELM-FBPINNs: An Efficient Multilevel Random Feature Method

Ben Moseley; Jennifer Pestana; Samuel Anderson; Victorita Dolean

arxiv: 2409.01949 · v3 · pith:5YFMISXYnew · submitted 2024-09-03 · 🧮 math.NA · cs.NA

ELM-FBPINNs: An Efficient Multilevel Random Feature Method

Samuel Anderson , Victorita Dolean , Ben Moseley , Jennifer Pestana This is my paper

Pith reviewed 2026-05-23 21:24 UTC · model grok-4.3

classification 🧮 math.NA cs.NA

keywords extreme learning machinesphysics-informed neural networksdomain decompositionrandom featuresPINNsFBPINNsmultilevel methodsPDE solvers

0 comments

The pith

ELM-FBPINNs replace trainable subdomain networks in finite basis PINNs with extreme learning machines, reducing PDE training to a structured linear least-squares problem.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes a hybrid method that combines multilevel domain decomposition and partition-of-unity constructions with random feature models. This replaces the iterative nonlinear optimization inside each subdomain of FBPINNs with extreme learning machines. Training then becomes a linear least-squares problem with no backpropagation required. Systematic comparisons on benchmark PDEs show that the resulting ELM-FBPINNs and multilevel variants reach competitive accuracy while accelerating convergence and increasing robustness to choices of architecture and optimizer. Ablation experiments separate the contributions of domain decomposition from those of the random-feature enrichment.

Core claim

By integrating multilevel domain decomposition and partition-of-unity constructions with random feature models, the multilevel ELM-FBPINN replaces trainable subdomain networks with extreme learning machines. This eliminates backpropagation entirely and reduces training to a structured linear least-squares problem, while achieving competitive accuracy on representative benchmark problems compared to standard PINNs and FBPINNs.

What carries the argument

Extreme learning machines placed inside each subdomain of the multilevel finite-basis PINN framework, which reformulates the entire training task as a linear least-squares problem.

If this is right

ELM-FBPINNs achieve competitive accuracy while significantly accelerating convergence on benchmark problems.
Robustness to architectural and optimisation parameters improves relative to standard PINNs and FBPINNs.
Domain decomposition and random feature enrichment separately control expressivity, conditioning, and scalability.
Backpropagation is eliminated entirely, removing the need for iterative nonlinear solvers inside subdomains.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The linear least-squares structure may allow direct use of sparse or iterative linear solvers that scale to larger subdomain counts than nonlinear optimizers permit.
Random-feature enrichment could be inserted into other localized PINN variants to reduce per-subdomain training cost without altering the overall domain-decomposition layout.
Multilevel extensions of the same construction may further improve scalability for PDEs on complex geometries where single-level decompositions remain expensive.

Load-bearing premise

Extreme learning machines retain enough approximation power inside each subdomain that the resulting linear least-squares problem stays well-conditioned and does not lose expressivity relative to nonlinearly trained networks.

What would settle it

On a standard benchmark such as the Poisson equation or a nonlinear wave problem, the ELM-FBPINN either produces solution errors substantially larger than those of FBPINNs or yields an ill-conditioned least-squares matrix whose solution deviates markedly from the true PDE solution.

read the original abstract

Domain-decomposed variants of physics-informed neural networks (PINNs) such as finite basis PINNs (FBPINNs) mitigate some of PINNs' issues like slow convergence and spectral bias through localisation, but still rely on iterative nonlinear optimisation within each subdomain. In this work, we propose a hybrid approach that combines multilevel domain decomposition and partition-of-unity constructions with random feature models, yielding a method referred to as multilevel ELM-FBPINN. By replacing trainable subdomain networks with extreme learning machines, the resulting formulation eliminates backpropagation entirely and reduces training to a structured linear least-squares problem. We provide a systematic numerical study comparing ELM-FBPINNs and multilevel ELM-FBPINNs with standard PINNs and FBPINNs on representative benchmark problems, demonstrating that ELM-FBPINNs and multilevel ELM-FBPINNs achieve competitive accuracy while significantly accelerating convergence and improving robustness with respect to architectural and optimisation parameters. Through ablation studies, we further clarify the distinct roles of domain decomposition and random feature enrichment in controlling expressivity, conditioning, and scalability.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The paper swaps trainable subdomain networks for extreme learning machines inside multilevel FBPINNs, turning the problem into a single linear least-squares solve and reporting faster convergence on benchmarks.

read the letter

The central contribution is replacing the nonlinear optimization inside each subdomain of FBPINNs with random-feature extreme learning machines. This produces an explicit linear least-squares formulation that eliminates backpropagation while keeping the multilevel partition-of-unity structure. The authors run a systematic numerical comparison against plain PINNs and standard FBPINNs on several benchmark PDEs, plus ablation experiments that isolate the effects of domain decomposition versus random-feature enrichment. Those experiments are the part that actually adds value: they give concrete evidence that accuracy remains competitive while training becomes faster and less sensitive to hyperparameter choices such as feature count or subdomain layout. The formulation itself is straightforward and the linear-algebra step is reproducible in principle. The main soft spot is conditioning of the assembled least-squares matrix. The stress-test concern is reasonable: nothing in the abstract indicates a priori bounds or regularization, and if the reported runs do not track condition numbers as subdomain count or feature count grows, the robustness claim is limited to the specific test problems rather than a general property. The paper does not appear to contain internal contradictions or circular claims; the method is defined cleanly and the numerical claims are falsifiable. This is useful reading for anyone already working on domain-decomposed PINN variants who wants a faster training route without leaving the PINN framework. It is not paradigm-shifting, but the combination is new enough and the numerics are presented systematically enough that it merits a serious referee. I would send it to review and ask specifically for condition-number diagnostics and scaling tests on stiffer problems.

Referee Report

2 major / 0 minor

Summary. The manuscript proposes ELM-FBPINNs, a hybrid method that replaces the trainable subdomain networks in finite-basis PINNs (FBPINNs) with extreme learning machines (random feature models). This yields an explicit structured linear least-squares problem that eliminates backpropagation. The multilevel extension combines this with domain decomposition and partition-of-unity weighting. A systematic numerical study on benchmark PDEs is claimed to show that both ELM-FBPINNs and multilevel ELM-FBPINNs attain competitive accuracy while accelerating convergence and improving robustness to architectural and optimization parameters.

Significance. If the performance and robustness claims are substantiated, the approach would supply a computationally lighter alternative to standard PINNs and FBPINNs for localized PDE approximation, removing iterative nonlinear optimization while retaining the benefits of domain decomposition.

major comments (2)

[Abstract] Abstract: the central claim of 'competitive accuracy' and 'significantly accelerating convergence' is asserted on the basis of a systematic numerical study, yet the abstract (and the information supplied) contains no quantitative error values, convergence rates, error bars, or specific benchmark comparisons, preventing verification of the performance assertions.
[Numerical study] Numerical study / method formulation: the linear least-squares matrix assembled from random-feature residuals and partition-of-unity weights is asserted to remain well-conditioned and robust across the reported parameter ranges, but no condition-number measurements, regularization strategy, or a-priori bounds are indicated; without such evidence the robustness claim cannot be separated from the specific test problems chosen.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the constructive comments. We address each major point below.

read point-by-point responses

Referee: [Abstract] Abstract: the central claim of 'competitive accuracy' and 'significantly accelerating convergence' is asserted on the basis of a systematic numerical study, yet the abstract (and the information supplied) contains no quantitative error values, convergence rates, error bars, or specific benchmark comparisons, preventing verification of the performance assertions.

Authors: We agree that the abstract would be strengthened by including quantitative indicators. We will revise the abstract to report representative error values, convergence improvements, and benchmark comparisons from the numerical study. revision: yes
Referee: [Numerical study] Numerical study / method formulation: the linear least-squares matrix assembled from random-feature residuals and partition-of-unity weights is asserted to remain well-conditioned and robust across the reported parameter ranges, but no condition-number measurements, regularization strategy, or a-priori bounds are indicated; without such evidence the robustness claim cannot be separated from the specific test problems chosen.

Authors: The robustness is evidenced by consistent performance across the reported parameter sweeps in the numerical study. We acknowledge that explicit condition-number measurements would provide additional support. We will add condition-number results for representative matrices and clarify any regularization in the revised manuscript. revision: yes

Circularity Check

0 steps flagged

No circularity: method is a direct substitution yielding an explicit linear solve

full rationale

The paper defines ELM-FBPINNs by replacing the trainable subdomain networks of FBPINNs with extreme learning machines, which converts the subdomain training step into a single structured linear least-squares problem assembled from random-feature residuals and partition-of-unity weights. This substitution is stated explicitly in the abstract and does not rely on any fitted parameter being renamed as a prediction, nor on any self-citation chain that would render the central claim tautological. Performance assertions (competitive accuracy, accelerated convergence, improved robustness) are presented as outcomes of a subsequent numerical study on benchmark problems rather than as quantities derived from the formulation itself. No equations or uniqueness theorems are shown to collapse back onto the method's own inputs by construction.

Axiom & Free-Parameter Ledger

2 free parameters · 1 axioms · 0 invented entities

The approach rests on standard assumptions from random feature models and domain decomposition; no new entities are introduced. Free parameters such as feature count and subdomain layout are expected but not quantified in the abstract.

free parameters (2)

number of random features per subdomain
Controls expressivity of each ELM; value chosen per problem but not reported in abstract.
number of subdomains and levels
Determines decomposition granularity; selected for the benchmark problems.

axioms (1)

domain assumption The resulting linear least-squares problem remains well-conditioned for the chosen random features and subdomain partition.
Required for the ELM replacement to deliver stable solutions without additional regularization.

pith-pipeline@v0.9.0 · 5723 in / 1330 out tokens · 27762 ms · 2026-05-23T21:24:14.364276+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

16 extracted references · 16 canonical work pages

[1]

Journal of Machine Learning 1, 268--98 (2022)

Chen, J., Chi, X., E, W., Yang, Z.: Bridging traditional and machine learning-based algorithms for solving pdes: the random feature method. Journal of Machine Learning 1, 268--98 (2022)

work page 2022
[2]

Dolean, V., Heinlein, A., Mishra, S., Moseley, B.: Finite basis physics-informed neural networks as a S chwarz domain decomposition method. In: Z. Dost \'a l, T. Kozubek, A. Klawonn, U. Langer, L.F. Pavarino, J. S \'i stek, O.B. Widlund (eds.) Domain Decomposition Methods in Science and Engineering XXVII, pp. 165--172. Springer Nature Switzerland, Cham (2024)

work page 2024
[3]

Computer Methods in Applied Mechanics and Engineering 429, 117116 (2024)

Dolean, V., Heinlein, A., Mishra, S., Moseley, B.: Multilevel domain decomposition-based architectures for physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering 429, 117116 (2024)

work page 2024
[4]

Neurocomputing 70(1), 489--501 (2006)

Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: Theory and applications. Neurocomputing 70(1), 489--501 (2006). Neural Networks

work page 2006
[5]

IEEE Transactions on Neural Networks 9(5), 987--1000 (1998)

Lagaris, I.E., Likas, A., Fotiadis, D.I.: Artificial neural networks for solving ordinary and partial differential equations . IEEE Transactions on Neural Networks 9(5), 987--1000 (1998)

work page 1998
[6]

Moseley, B.: Physics-informed machine learning: from concepts to real-world applications . Ph.D. thesis, University of Oxford (2022)

work page 2022
[7]

Advances in Computational Mathematics 49(4), 62 (2023)

Moseley, B., Markham, A., Nissen-Meyer, T.: Finite basis physics-informed neural networks ( FBPINNs ): a scalable domain decomposition approach for solving differential equations. Advances in Computational Mathematics 49(4), 62 (2023)

work page 2023
[8]

Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F., Bengio, Y., Courville, A.: On the spectral bias of neural networks. In: K. Chaudhuri, R. Salakhutdinov (eds.) Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 97, pp. 5301--5310. PMLR (2019)

work page 2019
[9]

Journal of Computational physics 378, 686--707 (2019)

Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics 378, 686--707 (2019)

work page 2019
[10]

Computer Methods in Applied Mechanics and Engineering 384, 113938 (2021)

Wang, S., Wang, H., Perdikaris, P.: On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks . Computer Methods in Applied Mechanics and Engineering 384, 113938 (2021)

work page 2021
[11]

, " * write output.state after.block = add.period write newline

ENTRY address archive author booktitle chapter doi edition editor eid eprint howpublished institution journal key month note number organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all ...

work page
[12]

write newline

" write newline "" before.all 'output.state := FUNCTION add.period duplicate empty 'skip "." * add.blank if FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap duplicate "7" = swap duplicate "8" = swap "9" = or or or or or or or or or FUNCTION ...

work page
[13]

, " * write output.state after.block = add.period write newline

ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 'after.sentence := #3 '...

work page
[14]

write newline

" write newline "" before.all 'output.state := FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap duplicate "7" = swap duplicate "8" = swap "9" = or or or or or or or or or FUNCTION n.separate 't := "" #0 'numnames := t empty not t #-1 #1 subs...

work page
[15]

, " * write output.state after.block = add.period write newline

ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 'after.sentence := #3 '...

work page
[16]

write newline

" write newline "" before.all 'output.state := FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap duplicate "7" = swap duplicate "8" = swap "9" = or or or or or or or or or FUNCTION n.separate 't := "" #0 'numnames := t empty not t #-1 #1 subs...

work page

[1] [1]

Journal of Machine Learning 1, 268--98 (2022)

Chen, J., Chi, X., E, W., Yang, Z.: Bridging traditional and machine learning-based algorithms for solving pdes: the random feature method. Journal of Machine Learning 1, 268--98 (2022)

work page 2022

[2] [2]

Dolean, V., Heinlein, A., Mishra, S., Moseley, B.: Finite basis physics-informed neural networks as a S chwarz domain decomposition method. In: Z. Dost \'a l, T. Kozubek, A. Klawonn, U. Langer, L.F. Pavarino, J. S \'i stek, O.B. Widlund (eds.) Domain Decomposition Methods in Science and Engineering XXVII, pp. 165--172. Springer Nature Switzerland, Cham (2024)

work page 2024

[3] [3]

Computer Methods in Applied Mechanics and Engineering 429, 117116 (2024)

Dolean, V., Heinlein, A., Mishra, S., Moseley, B.: Multilevel domain decomposition-based architectures for physics-informed neural networks. Computer Methods in Applied Mechanics and Engineering 429, 117116 (2024)

work page 2024

[4] [4]

Neurocomputing 70(1), 489--501 (2006)

Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: Theory and applications. Neurocomputing 70(1), 489--501 (2006). Neural Networks

work page 2006

[5] [5]

IEEE Transactions on Neural Networks 9(5), 987--1000 (1998)

Lagaris, I.E., Likas, A., Fotiadis, D.I.: Artificial neural networks for solving ordinary and partial differential equations . IEEE Transactions on Neural Networks 9(5), 987--1000 (1998)

work page 1998

[6] [6]

Moseley, B.: Physics-informed machine learning: from concepts to real-world applications . Ph.D. thesis, University of Oxford (2022)

work page 2022

[7] [7]

Advances in Computational Mathematics 49(4), 62 (2023)

Moseley, B., Markham, A., Nissen-Meyer, T.: Finite basis physics-informed neural networks ( FBPINNs ): a scalable domain decomposition approach for solving differential equations. Advances in Computational Mathematics 49(4), 62 (2023)

work page 2023

[8] [8]

Rahaman, N., Baratin, A., Arpit, D., Draxler, F., Lin, M., Hamprecht, F., Bengio, Y., Courville, A.: On the spectral bias of neural networks. In: K. Chaudhuri, R. Salakhutdinov (eds.) Proceedings of the 36th International Conference on Machine Learning, Proceedings of Machine Learning Research, vol. 97, pp. 5301--5310. PMLR (2019)

work page 2019

[9] [9]

Journal of Computational physics 378, 686--707 (2019)

Raissi, M., Perdikaris, P., Karniadakis, G.E.: Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational physics 378, 686--707 (2019)

work page 2019

[10] [10]

Computer Methods in Applied Mechanics and Engineering 384, 113938 (2021)

Wang, S., Wang, H., Perdikaris, P.: On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks . Computer Methods in Applied Mechanics and Engineering 384, 113938 (2021)

work page 2021

[11] [11]

, " * write output.state after.block = add.period write newline

ENTRY address archive author booktitle chapter doi edition editor eid eprint howpublished institution journal key month note number organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all ...

work page

[12] [12]

write newline

" write newline "" before.all 'output.state := FUNCTION add.period duplicate empty 'skip "." * add.blank if FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap duplicate "7" = swap duplicate "8" = swap "9" = or or or or or or or or or FUNCTION ...

work page

[13] [13]

, " * write output.state after.block = add.period write newline

ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 'after.sentence := #3 '...

work page

[14] [14]

write newline

" write newline "" before.all 'output.state := FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap duplicate "7" = swap duplicate "8" = swap "9" = or or or or or or or or or FUNCTION n.separate 't := "" #0 'numnames := t empty not t #-1 #1 subs...

work page

[15] [15]

, " * write output.state after.block = add.period write newline

ENTRY address author booktitle chapter doi edition editor eid howpublished institution journal key month note number organization pages publisher school series title type url volume year label INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 'after.sentence := #3 '...

work page

[16] [16]

write newline

" write newline "" before.all 'output.state := FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap duplicate "7" = swap duplicate "8" = swap "9" = or or or or or or or or or FUNCTION n.separate 't := "" #0 'numnames := t empty not t #-1 #1 subs...

work page