Pith. sign in

REVIEW 3 major objections 4 minor 93 references

Leveraging Operator Learning to Accelerate Convergence of the Preconditioned Conjugate Gradient Method

T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read DeepONet builds deflation spaces that cut PCG iteration counts

desk verdict A useful empirical study of DeepONet-based deflation for PCG showing consistent iteration gains, but the random trunk-basis selection needs a control before the operator-learning contribution is fully convincing. read the letter →

arxiv 2508.00101 v1 pith:LGCADI7L submitted 2025-07-31 math.NA cs.LGcs.NAmath.OC

Alena Kopaničáková , Youngkyu Lee , George Em Karniadakis This is my paper
classification math.NAcs.LGcs.NAmath.OC MSC 65F1065F0865N2268T07
keywords deflationpreconditionedconjugategradientDeepONetoperatorlearningparametricPDEscoarsespaceKrylovsubspacerecyclingdomaindecomposition
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that a trained DeepONet can supply high-quality deflation vectors for the preconditioned conjugate gradient method, substantially reducing the number of iterations needed to solve sequences of parametric linear systems. Two complementary construction routes are proposed: one uses the trunk basis functions of the DeepONet as approximations of the near-null space of the discrete operator, and the other uses DeepONet-predicted solutions as recycled Krylov directions. The paper reports that both approaches consistently beat the classical near-null-space (NICO) deflation vectors across steady-state, time-dependent, scalar, and vector-valued benchmark problems, and that the gains persist when the mesh is refined far beyond the training resolution. If correct, this gives a practical way to build effective coarse spaces for ill-conditioned parametric PDE systems without solving eigenvalue problems.

What carries the argument

The deflation projector is the standard two-level construct $\Pi = I - C A$ with $C = P A_c^{-1} R$, $A_c = R A P$, where the prolongation operator $P$ is assembled from vectors extracted from a trained DeepONet. In the TB approach each column of the tentative $\tilde{P}$ is a trunk-network output $T_r(x_j)$ evaluated at all mesh nodes, chosen by random index selection; in the RS approach each column is a DeepONet prediction $\sum_k \prod_b B^b_k(y^b) T_k(x)$ for the current or randomly sampled branch inputs. The tentative operator is then partitioned into blocks according to user-chosen groups of degrees of freedom, and each block is orthonormalized via QR factorization to produce the final block-diagonal $P$. This machinery converts the task of finding near-null-space vectors into the task of evaluating a trained network, and it preserves the standard DPCG convergence theory because only the transfer operators change.

What would settle it

Apply the TB-deflated PCG to a parametric Helmholtz-like operator whose near-null space consists of highly oscillatory modes: if the randomly selected trunk basis functions do not align with those modes, the iteration count will not improve over NICO deflation and the central claim of consistent outperformance would be refuted. A simpler check is to compare random trunk selection against explicit SVD-based selection of the lowest-singular-value trunk functions on one of the paper's own benchmarks; a large gap in iteration counts would show that random selection is not a reliable proxy.

Watch

Extended reading notes

Core claim

The central claim is that the deflation subspace for PCG can be generated by a pre-trained operator network, specifically DeepONet, in two distinct ways: (i) the trunk basis (TB) approach, which evaluates randomly selected trunk-network basis functions at all mesh nodes to form deflation vectors that approximate near-null-space modes; and (ii) the recycled solution (RS) approach, which forms deflation vectors from DeepONet predictions for the current parameter and for randomly sampled parameters. The paper argues that these learned deflation spaces capture the unfavorable spectral components of the preconditioned operator more effectively than textbook null-space vectors, especially for problems such as Darcy flow with jumping coefficients where classical near-null-space information is poor. The authors further show that imposing a block structure on the deflation operator by grouping degrees of freedom--through problem knowledge, preconditioner subdomains, or k-means clustering of the predicted solution--improves the trade-off between iteration reduction and per-iteration cost. Across all reported experiments, the DeepONet-based deflated PCG reduces iteration counts compared to both undeflated PCG and NICO-deflated PCG, and it remains effective when the test mesh is several refinement levels finer than the training mesh.

Load-bearing premise

In the TB variant, the load-bearing premise is that randomly selected trunk basis functions of the trained DeepONet approximate near-null-space vectors of the discretized parametric operator; the paper supports this only by numerical experience rather than spectral analysis or an ablation versus SVD-based selection.

Editorial extensions

If this is right

  • PCG can be accelerated for parametric PDE families without computing eigenvectors or solving local eigenvalue problems, since the deflation space comes from a single offline-trained network.
  • Problems with jumping coefficients or otherwise unknown near-null spaces, where classical NICO deflation is weak or fails, become amenable to effective deflation using learned vectors.
  • The additive Schwarz preconditioner becomes algorithmically scalable when augmented with DeepONet-based deflation, with iteration counts that no longer grow with the number of subdomains.
  • For time-dependent problems the same trained network supplies deflation vectors at every time step, so the iteration savings accumulate over the whole simulation.
  • The trained network generalizes across mesh resolutions: a DeepONet trained on the coarsest mesh still yields effective deflation spaces on much finer meshes, decoupling the offline training cost from the online solve resolution.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If random selection of trunk basis functions is truly as effective as SVD-based selection, then the method's robustness rests on the trunk network's spectral content being diffuse enough that any low-singularity mode is a decent near-null-space candidate; this could be tested by comparing random selection against explicit smallest-singular-value selection on a single hard problem.
  • The clustering-based grouping strategy suggests a general recipe: use the network's predicted solution field to infer the problem's heterogeneity, then align deflation blocks with that heterogeneity; this may extend to problems with no natural subdomain structure.
  • The same deflation construction could plausibly be ported to other operator-learning backbones such as Fourier neural operators or transformer-based architectures, provided their output bases or predicted solutions can be evaluated pointwise on the discretization mesh.
  • The break-even analysis implies a practical stopping rule: the number of deflation vectors should be tuned per preconditioner and problem class by measuring the average relative iteration reduction per vector, rather than fixed a priori.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes two mechanisms for constructing deflation subspaces for the preconditioned conjugate gradient method from a pretrained DeepONet: a trunk-basis (TB) approach, in which selected trunk network outputs are used as deflation vectors, and a recycled-solution (RS) approach, in which DeepONet-predicted solutions for randomly sampled parameters are used. Both mechanisms are combined with strategies for imposing block sparsity on the deflation operator, either from problem structure, from the preconditioner's domain decomposition, or from k-means clustering of the predicted solution. The method is tested on six parametric PDE benchmarks (Darcy, JumpDarcy, linear elasticity in 3D, heat equation on structured and unstructured domains) with SSOR, ICC, and ASM preconditioners, and with varying numbers of deflation vectors and groups. The reported iteration counts show that TB and RS consistently reduce iteration counts relative to plain PCG and to the classical NICO deflation approach, and the paper claims generalization across parameters and mesh resolutions.

Significance. If the empirical claims are fully substantiated, the paper would introduce a useful new class of learned coarse spaces for parametric SPD systems, particularly for problems where classical near-null-space vectors are ineffective. The benchmark suite is comprehensive, the algorithm is described clearly, dataset sizes and training details are given, and the comparison against NICO is a sensible baseline. The main strengths are the breadth of the numerical study and the transparent presentation of the deflation construction. However, the current evidence is limited to iteration counts on a single mesh per benchmark; no wall-clock times or total computational costs are reported, the key mechanism behind the TB variant is not isolated by ablation, and the claimed resolution generalization is not demonstrated by any reported experiment.

major comments (3)
  1. [Section 3.2.1, Eq. (22)] The TB approach rests on the assertion that randomly selected trunk basis functions approximate near-null-space vectors of the discrete parametric operator, but the only support is Figure 2 (singular-value decay) and the statement that 'our numerical experience suggests' random selection is competitive. Smoothness of the trunk functions does not by itself imply near-null-space membership for a particular A(theta). Because the paper explicitly defers theoretical analysis to future work, an empirical control is essential. Please add an ablation comparing (i) SVD-based selection of low-singular-value trunk functions, (ii) random trunk functions, and (iii) a generic smooth basis of the same size (e.g., low-frequency Fourier modes or piecewise-linear coarse basis functions), with matched k and S, for at least one representative problem. Without such a control, the central claim that operator learning, rather than mere smoothness of the coarse basis, drives the observed convergence improvement is not established.
  2. [Section 5 and Section 6] The abstract and the summary claim generalization across 'problem resolutions', but no reported experiment varies the mesh resolution. All numerical tables give results for a single mesh per benchmark (T7 for Darcy, T6 for JumpDarcy, T5 for Heat and Snowflake, T4 for Wrench, T3 for E-shape), and the figures use fixed resolutions. The statement in Section 5 that the hierarchy T1,...,TL is used for testing is not supported by any visible result. Please include a resolution sweep (e.g., iterations versus mesh level for each benchmark) or explicitly remove the resolution-generalization claim.
  3. [Section 3.3 and Section 5] The paper's practical claim of 'accelerating convergence' is supported only by iteration counts. The per-iteration cost of deflation grows with k and S through the coarse-space solve, the projections involving P, and the block QR construction, but no wall-clock times, flop counts, or total matvec/coarse-solve counts are reported. The break-even analysis in Section 3.3 assumes a linear iteration-reduction model N_DPCG(k) ≈ (1 − theta k) N_PCG and uses illustrative values of theta rather than measured overheads for the actual configurations. To substantiate the acceleration claim, please report at least one wall-clock or total-cost measure (e.g., total solve time or total number of matrix-vector products plus coarse solves) for the main benchmark configurations.
minor comments (4)
  1. [Section 1] There is a duplicated word in 'To to achieve this goal'.
  2. [Section 3.3] The text says 'only trunk network (TS)' but the abbreviation used elsewhere is TB; please use TB consistently.
  3. [Section 5.2] The statement that NICO 'fails to produce a suitable coarse space' for JumpDarcy is stronger than the data support: in Tables 2 and 3 NICO converges for JumpDarcy, albeit with more iterations than TB and RS; the dash entries occur for Heat and Snowflake at larger S. Please phrase this as NICO being markedly less effective rather than failing for JumpDarcy.
  4. [Tables 1-3] All iteration counts are reported as averages over 10 runs without standard deviations, minima, or maxima. Since the TB and RS constructions involve random selection or random branch sampling, seed-to-seed variability is relevant to the claim of consistent improvement; adding error bars or range information would strengthen the comparison.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: the DPCG results are empirical, benchmarked against external NICO/PCG baselines, and the DeepONet deflation vectors are not fitted to the reported iteration counts.

full rationale

The paper derives no quantity by construction from its own inputs: the deflation vectors in (22) and (24) are trunk-basis evaluations and DeepONet solution predictions, respectively, and the reported iteration reductions are measured against the external NICO and undeflated PCG baselines in Tables 1-3. The DeepONet is trained on coarse-mesh solutions of the same parametric family, so the RS/TB vectors are surrogate-based coarse-space vectors rather than independent first-principles predictions; this limits the scientific weight of the generalization claim but is not circular reasoning. The central TB construction is attributed to the authors' prior work [49], a self-citation, but the paper supports it with its own Figure 2 and experiments, so the citation is not the sole load-bearing evidence. The random-selection justification ('our numerical experience suggests...') is an unproven empirical assertion, a correctness risk, not a circular reduction. No equation in the paper is equal by definition to a fitted parameter, and no uniqueness claim is imported from the authors. Hence score 1.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The method's central mechanism is empirical: a fitted DeepONet provides deflation vectors and group structure. No free parameters are derived from first principles; k, S, and the network weights are chosen or trained. The key unproved assumptions are the near-null-space fidelity of randomly selected trunk functions and the transfer of a coarse-mesh-trained network to finer meshes.

free parameters (4)
  • DeepONet parameters w = Trained with Adam (lr=1e-4, batch=1000, early stopping after 10k epochs) on 2,500 to 10,000 coarse-mesh solution…
    TB and RS deflation vectors are direct outputs of this fitted model, so the method's quality depends on fitting quality.
  • Number of deflation vectors k = 5 for scalar problems, 16 or 24 for heat and vector-valued problems
    User-chosen; Section 5.1 shows strong sensitivity of iteration counts to k.
  • Number of dof groups S = 4 to 256
    Sets the block structure and coarse-space dimension; Section 5.2 shows only modest iteration gains with larger S.
  • Trunk latent dimension p = 128 everywhere
    Size of the trunk basis pool for the TB approach; fixed without a reported sensitivity study.
assumptions (5)
  • standard math The DPCG error bound (Eq. 13) and the orthogonality conditions of Algorithm 1 hold for the constructed block-diagonal deflation operator P.
    Standard deflation theory from [79,89]; the paper assumes it applies because P has full column rank and Ac = RAP is invertible.
  • ad hoc to paper Near-null-space vectors of the parametric discrete operator are well approximated by DeepONet trunk basis functions, and random selection among them is competitive with SVD-based selection.
    Section 3.2.1, Eq. (22); supported only by 'numerical experience', with no spectral measurement or ablation.
  • domain assumption DeepONet-predicted solutions for the current and randomly sampled parameter vectors span a subspace that contains the slow modes of M A.
    Section 3.2.1 RS approach; a transfer of the standard recycling assumption [17,79] to learned predictions, not verified spectrally.
  • domain assumption A DeepONet trained on the coarsest mesh T1 transfers to finer meshes T2-T7 and unseen parameter samples.
    Section 4.4 and Section 5; demonstrated empirically for each benchmark but no theoretical guarantee, and the paper lists this analysis as future work.
  • ad hoc to paper k-means clustering of the DeepONet-predicted solution groups dofs with similar spectral behavior.
    Section 3.2.2 item 3; no analysis of why clustering solution magnitudes yields effective coarse spaces.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Leveraging Operator Learning to Accelerate Convergence of the Preconditioned Conjugate Gradient Method." pith.science (2026). https://pith.science/paper/LGCADI7L

@misc{pith2026250800101,
  author       = {Pith},
  title        = {Pith review of: Leveraging Operator Learning to Accelerate Convergence of the Preconditioned Conjugate Gradient Method},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LGCADI7L}},
  note         = {Machine review of arXiv:2508.00101}
}
read the original abstract

We propose a new deflation strategy to accelerate the convergence of the preconditioned conjugate gradient(PCG) method for solving parametric large-scale linear systems of equations. Unlike traditional deflation techniques that rely on eigenvector approximations or recycled Krylov subspaces, we generate the deflation subspaces using operator learning, specifically the Deep Operator Network~(DeepONet). To this aim, we introduce two complementary approaches for assembling the deflation operators. The first approach approximates near-null space vectors of the discrete PDE operator using the basis functions learned by the DeepONet. The second approach directly leverages solutions predicted by the DeepONet. To further enhance convergence, we also propose several strategies for prescribing the sparsity pattern of the deflation operator. A comprehensive set of numerical experiments encompassing steady-state, time-dependent, scalar, and vector-valued problems posed on both structured and unstructured geometries is presented and demonstrates the effectiveness of the proposed DeepONet-based deflated PCG method, as well as its generalization across a wide range of model parameters and problem resolutions.

Figures

Figures reproduced from arXiv: 2508.00101 by the authors.

Figure 1
Figure 1. Left: An example of the multi-input DeepONet [61]. Right: Examples of two groups, [PITH_FULL_IMAGE:figures/full_fig_p008_1.png] view at source ↗
Figure 2
Figure 2. Left: Visualization of singular values of TB functions extracted from DeepONet ( [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. A sketch of the computational pipeline for DPCG with a DeepONet-induced deflation [PITH_FULL_IMAGE:figures/full_fig_p012_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Left: An illustration of the spatially varying branch input features ( [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Left: An illustration of the computational domain with channel patterns used for the [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: A simulation result of a 3D wrench problem under loading with [PITH_FULL_IMAGE:figures/full_fig_p015_6.png]
Figure 7
Figure 7. Figure 7: An example of a result of a 3D E-shape simulation results for varying material parameters. [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: The average number of iterations required by the DPCG method with RS deflation to [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: The average number of iterations required by the DPCG method to reach convergence [PITH_FULL_IMAGE:figures/full_fig_p018_9.png]
Figure 10
Figure 10. Figure 10: The average number of iterations required by the DPCG method to reach the convergence [PITH_FULL_IMAGE:figures/full_fig_p022_10.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

93 extracted references · 68 canonical work pages

  1. [49]

    DeepONet based preconditioning strategies for solving parametric linear systems of equations

    Alena Kopaničáková and George Em Karniadakis. DeepONet based preconditioning strategies for solving parametric linear systems of equations. SIAM Journal on Scientific Computing, 47(1):C151–C181, 2025

    work page 2025

  2. [1]

    Machine-learned preconditioners for linear solvers in geophysical fluid flows.arXiv:2010.02866, 2020

    Jan Ackmann, Peter D Düben, Tim N Palmer, and Piotr K Smolarkiewicz. Machine-learned preconditioners for linear solvers in geophysical fluid flows.arXiv:2010.02866, 2020

    arXiv 2010

  3. [2]

    The k-means algo- rithm: A comprehensive survey and performance evaluation.Electronics, 9(8):1295, 2020

    Mohiuddin Ahmed, Raihan Seraj, and Syed Mohammed Shamsul Islam. The k-means algo- rithm: A comprehensive survey and performance evaluation.Electronics, 9(8):1295, 2020

    2020

  4. [3]

    Acceleratingalgebraicmultigridmethods via artificial neural networks.Vietnam Journal of Mathematics, 51(1):1–36, 2023

    PaolaFAntonietti, MatteoCaldana, andLucaDede’. Acceleratingalgebraicmultigridmethods via artificial neural networks.Vietnam Journal of Mathematics, 51(1):1–36, 2023

    2023

  5. [4]

    Principled acceleration of iterative numerical methods using machine learning

    Sohei Arisaka and Qianxiao Li. Principled acceleration of iterative numerical methods using machine learning. In Proceedings of the 40th International Conference on Machine Learning, volume 202 ofProceedings of Machine Learning Research, pages 1041–1059. PMLR, 23–29 Jul 2023

    2023

  6. [5]

    Multigrid-augmented deep learning preconditioners for the Helmholtz equation

    Yael Azulay and Eran Treister. Multigrid-augmented deep learning preconditioners for the Helmholtz equation. SIAM Journal on Scientific Computing, 45(3):S127–S151, 2023

    2023

  7. [6]

    PETSc users manual

    Satish Balay, Shrirang Abhyankar, Mark Adams, Jed Brown, Peter Brune, Kris Buschelman, Lisandro Dalcin, Alp Dener, Victor Eijkhout, W Gropp, et al. PETSc users manual. 2019

    2019

  8. [7]

    Appli- cation of machine learning to the selection of sparse linear solvers.Int

    Sanjukta Bhowmick, Victor Eijkhout, Yoav Freund, Erika Fuentes, and David Keyes. Appli- cation of machine learning to the selection of sparse linear solvers.Int. J. High Perf. Comput. Appl, 2006

    2006

Show all 93 references
  1. [8]

    SIAM, Philadelphia, 2000

    William L Briggs, Steve F McCormick, et al.A multigrid tutorial. SIAM, Philadelphia, 2000

    2000

  2. [9]

    A deflation technique for linear systems of equations.SIAM Journal on Scientific Computing, 19(4):1245–1260, 1998

    Kevin Burrage, Jocelyne Erhel, Bert Pohl, and Alan Williams. A deflation technique for linear systems of equations.SIAM Journal on Scientific Computing, 19(4):1245–1260, 1998

    1998

  3. [10]

    Krylov-subspace recycling via the POD- augmented conjugate-gradient method

    Kevin Carlberg, Virginia Forstall, and Ray Tuminaro. Krylov-subspace recycling via the POD- augmented conjugate-gradient method. SIAM Journal on Matrix Analysis and Applications, 37(3):1304–1336, 2016. 25

    2016

  4. [11]

    Deflated and augmented Krylov subspace techniques

    Andrew Chapman and Yousef Saad. Deflated and augmented Krylov subspace techniques. Numerical Linear Algebra with Applications, 4(1):43–66, 1997

    1997

  5. [12]

    Meta-MgNet: Meta multigrid networks for solving parameterized partial differential equations

    Yuyan Chen, Bin Dong, and Jinchao Xu. Meta-MgNet: Meta multigrid networks for solving parameterized partial differential equations. Journal of Computational Physics, 455:110996, 2022

    2022

  6. [13]

    Learningadaptivecoarsespacesof BDDC algorithms for stochastic elliptic problems with oscillatory and high contrast coefficients

    EricChung, Hyea-HyunKim, Ming-FaiLam, andLinaZhao. Learningadaptivecoarsespacesof BDDC algorithms for stochastic elliptic problems with oscillatory and high contrast coefficients. Mathematical and Computational Applications, 26(2):44, 2021

    2021

  7. [14]

    Spectralcoarsespacesforthesubstructuredparallel Schwarz method.Journal of Scientific Computing, 91(3):69, 2022

    GabrieleCiaramellaandTommasoVanzan. Spectralcoarsespacesforthesubstructuredparallel Schwarz method.Journal of Scientific Computing, 91(3):69, 2022

    2022

  8. [15]

    On POD-based deflation vectors for DPCG applied to porous media problems.Journal of Computational and Applied Mathematics, 330:193–213, 2018

    GB Diaz Cortes, Cornelis Vuik, and Jan Dirk Jansen. On POD-based deflation vectors for DPCG applied to porous media problems.Journal of Computational and Applied Mathematics, 330:193–213, 2018

    2018

  9. [16]

    Fourier neural solver for large sparse linear algebraic systems

    Chen Cui, Kai Jiang, Yun Liu, and Shi Shu. Fourier neural solver for large sparse linear algebraic systems. Mathematics, 10(21):4014, 2022

    2022

  10. [17]

    Recycling krylov sub- spaces and truncating deflation subspaces for solving sequence of linear systems.ACM Trans- actions on Mathematical Software (TOMS), 47(2):1–30, 2021

    Hussam Al Daas, Laura Grigori, Pascal Hénon, and Philippe Ricoux. Recycling krylov sub- spaces and truncating deflation subspaces for solving sequence of linear systems.ACM Trans- actions on Mathematical Software (TOMS), 47(2):1–30, 2021

    2021

  11. [18]

    Accelerating the solution of linear systems appearing in two-phase reservoir simulation by the use of pod-based deflation methods

    Gabriela Berenice Diaz Cortés, Cornelis Vuik, and Jan-Dirk Jansen. Accelerating the solution of linear systems appearing in two-phase reservoir simulation by the use of pod-based deflation methods. Computational Geosciences, 25(5):1621–1645, 2021

    2021

  12. [19]

    Numerical solution of mixed- dimensional pdes using a neural preconditioner.arXiv preprint arXiv:2505.08491, 2025

    Nunzio Dimola, Nicola Rares Franco, and Paolo Zunino. Numerical solution of mixed- dimensional pdes using a neural preconditioner.arXiv preprint arXiv:2505.08491, 2025

    2025 arXiv

  13. [20]

    SIAM, Philadelphia, 2015

    Victorita Dolean, Pierre Jolivet, and Frédéric Nataf.An introduction to domain decomposition methods: algorithms, theory, and parallel implementation. SIAM, Philadelphia, 2015

    2015

  14. [21]

    A recursively recurrent neural network (R2N2) architecture for learning iterative algorithms.SIAM Journal on Scientific Computing, 46(2):A719–A743, 2024

    Danimir T Doncevic, Alexander Mitsos, Yue Guo, Qianxiao Li, Felix Dietrich, Manuel Dahmen, and Ioannis G Kevrekidis. A recursively recurrent neural network (R2N2) architecture for learning iterative algorithms.SIAM Journal on Scientific Computing, 46(2):A719–A743, 2024

    2024

  15. [22]

    Conjugate gradient method with preconditioning by projector.International Journal of Computer Mathematics, 23(3-4):315–323, 1988

    Zdeněk Dostál. Conjugate gradient method with preconditioning by projector.International Journal of Computer Mathematics, 23(3-4):315–323, 1988

    1988

  16. [24]

    On the construction of deflation-based preconditioners.SIAM Journal on Scientific Computing, 23(2):442–462, 2001

    Jason Frank and Cornelis Vuik. On the construction of deflation-based preconditioners.SIAM Journal on Scientific Computing, 23(2):442–462, 2001

    2001

  17. [25]

    Neural network preconditioning: a case study for the solution of the parametric Helmholtz equation

    Luc Giraud, Carola Kruse, Paul Mycek, Maksym Shpakovych, and Yanfei Xiang. Neural network preconditioning: a case study for the solution of the parametric Helmholtz equation. PhD thesis, Inria Centre at the University of Bordeaux, France, 2025. 26

    2025

  18. [26]

    Springer International Publishing, Cham, 2023

    Somdatta Goswami, Aniruddha Bora, Yue Yu, and George Em Karniadakis.Physics-Informed Deep Neural Operator Networks, pages 219–254. Springer International Publishing, Cham, 2023

    2023

  19. [27]

    Machine learning-aided numerical linear algebra: Convo- lutional neural networks for the efficient preconditioner generation

    Markus Götz and Hartwig Anzt. Machine learning-aided numerical linear algebra: Convo- lutional neural networks for the efficient preconditioner generation. In2018 IEEE/ACM 9th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems (scalA), pages 49–56. IEEE, 2018

    2018

  20. [28]

    Recursive bound-constrained Ada- Grad with applications to multilevel and domain decomposition minimization.arXiv preprint arXiv:2507.11513, 2025

    Serge Gratton, Alena Kopaničáková, and Philippe Toint. Recursive bound-constrained Ada- Grad with applications to multilevel and domain decomposition minimization.arXiv preprint arXiv:2507.11513, 2025

    2025 arXiv

  21. [29]

    Serge Gratton, Alena Kopaničáková, and Philippe L. Toint. Multilevel objective-function-free optimization with an application to neural networks training.SIAM Journal on Optimization, 33(4):2772–2800, 2023

    2023

  22. [30]

    Performance pre- diction of multigrid-solver configurations

    Alexander Grebhahn, Norbert Siegmund, Harald Köstler, and Sven Apel. Performance pre- diction of multigrid-solver configurations. InSoftware for Exascale Computing-SPPEXA 2013- 2015, pages 69–88. Springer, 2016

    2013

  23. [31]

    Iterative Methods for Solving Linear Systems

    Anne Greenbaum. Iterative Methods for Solving Linear Systems. SIAM, Philadelphia, 1997

    1997

  24. [32]

    Learning to optimize multigrid PDE solvers

    Daniel Greenfeld, Meirav Galun, Ronen Basri, Irad Yavneh, and Ron Kimmel. Learning to optimize multigrid PDE solvers. In International Conference on Machine Learning, pages 2415–2423. PMLR, 2019

    2019

  25. [33]

    Machine learning in adaptive domain decomposition methods—predicting the geometric location of constraints

    Alexander Heinlein, Axel Klawonn, Martin Lanser, and Janine Weber. Machine learning in adaptive domain decomposition methods—predicting the geometric location of constraints. SIAM Journal on Scientific Computing, 41(6):A3887–A3912, 2019

    2019

  26. [34]

    Combining machine learning and domain decomposition methods for the solution of partial differential equations–a review

    Alexander Heinlein, Axel Klawonn, Martin Lanser, and Janine Weber. Combining machine learning and domain decomposition methods for the solution of partial differential equations–a review. GAMM-Mitteilungen, 44(1):e202100001, 2021

    2021

  27. [35]

    Methods of conjugate gradients for solving linear systems

    Magnus R Hestenes, Eduard Stiefel, et al. Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau of Standards, 49(6):409–436, 1952

    1952

  28. [36]

    Meta-learning in neural networks: A survey

    Timothy Hospedales, Antreas Antoniou, Paul Micaelli, and Amos Storkey. Meta-learning in neural networks: A survey. IEEE transactions on pattern analysis and machine intelligence, 44(9):5149–5169, 2021

    2021

  29. [37]

    Learn- ing neural PDE solvers with convergence guarantees

    Jun-Ting Hsieh, Shengjia Zhao, Stephan Eismann, Lucia Mirabella, and Stefano Ermon. Learn- ing neural PDE solvers with convergence guarantees. InInternational Conference on Learning Representations, 2019

    2019

  30. [38]

    Int-deep: Adeeplearninginitializediterative method for nonlinear problems.Journal of Computational Physics, 419:109675, 2020

    JianguoHuang, HaoqinWang, andHaizhaoYang. Int-deep: Adeeplearninginitializediterative method for nonlinear problems.Journal of Computational Physics, 419:109675, 2020

    2020

  31. [39]

    Learning optimal multigrid smoothers via neural networks

    Ru Huang, Ruipeng Li, and Yuanzhe Xi. Learning optimal multigrid smoothers via neural networks. SIAM Journal on Scientific Computing, (0):S199–S225, 2022. 27

    2022

  32. [40]

    MIONet: Learning multiple-input operators via tensor product

    Pengzhan Jin, Shuai Meng, and Lu Lu. MIONet: Learning multiple-input operators via tensor product. SIAM Journal on Scientific Computing, 44(6):A3490–A3514, 2022

    2022

  33. [41]

    On the use of rigid body modes in the deflated preconditioned conjugate gradient method.SIAM Journal on Scientific Computing, 35(1):B207–B225, 2013

    TB Jonsthovel, Martin B van Gijzen, Cornelis Vuik, and A Scarpas. On the use of rigid body modes in the deflated preconditioned conjugate gradient method.SIAM Journal on Scientific Computing, 35(1):B207–B225, 2013

    2013

  34. [42]

    On the geometry transferability of the hybrid iterative numerical solver for differ- ential equations.Computational Mechanics, 72(3):471–484, 2023

    Adar Kahana, Enrui Zhang, Somdatta Goswami, George Karniadakis, Rishikesh Ranade, and Jay Pathak. On the geometry transferability of the hybrid iterative numerical solver for differ- ential equations.Computational Mechanics, 72(3):471–484, 2023

    2023

  35. [43]

    A deep conjugate direction method for iteratively solving linear systems

    Ayano Kaneda, Osman Akar, Jingyu Chen, Victoria Alicia Trevino Kala, David Hyde, and Joseph Teran. A deep conjugate direction method for iteratively solving linear systems. In Proceedings of the 40th International Conference on Machine Learning, volume 202 ofProceed- ings of M...

    2023

  36. [44]

    George Karypis and Vipin Kumar. METIS: A software package for partitioning unstructured graphs, partitioning meshes, and computing fill-reducing orderings of sparse matrices.Retrieved from the University Digital Conservancy, 1997

    1997

  37. [45]

    Black-box learning of multigrid parameters

    Alexandr Katrutsa, Talgat Daulbaev, and Ivan Oseledets. Black-box learning of multigrid parameters. Journal of Computational and Applied Mathematics, 368:112524, 2020

    2020

  38. [46]

    Learning adaptive FETI-DP constraints for irregular domain decompositions

    Axel Klawonn, Martin Lanser, and Janine Weber. Learning adaptive FETI-DP constraints for irregular domain decompositions. InDomain Decomposition Methods in Science and Engineer- ing XXVII, pages 279–286, Cham, 2024. Springer Nature Switzerland

    2024

  39. [47]

    Sur une courbe continue sans tangente, obtenue par une construction géométrique élémentaire

    HV Koch. Sur une courbe continue sans tangente, obtenue par une construction géométrique élémentaire. Arkiv for Matematik, Astronomi och Fysik, 1:681–704, 1904

    1904

  40. [48]

    DONprecond: Deep operator learning preconditioning strategies

    Alena Kopaničáková. DONprecond: Deep operator learning preconditioning strategies. Git repository, 2023

    2023

  41. [50]

    Enhancing training of physics-informed neural networks using domain decomposition–based precondition- ing strategies

    Alena Kopaničáková, Hardik Kothari, George E Karniadakis, and Rolf Krause. Enhancing training of physics-informed neural networks using domain decomposition–based precondition- ing strategies. SIAM Journal on Scientific Computing, 46(5):S46–S67, 2024

    2024

  42. [51]

    On weight initialization in deep neural networks.arXiv preprint arXiv:1704.08863, 2017

    Siddharth Krishna Kumar. On weight initialization in deep neural networks.arXiv preprint arXiv:1704.08863, 2017

    2017 arXiv

  43. [52]

    A nonoverlapping domain decomposition method for extreme learning machines: Elliptic problems

    Chang-Ock Lee, Youngkyu Lee, and Byungeun Ryoo. A nonoverlapping domain decomposition method for extreme learning machines: Elliptic problems. Computers & Mathematics with Applications, 189:109–128, 2025

    2025

  44. [53]

    Two-level overlapping addi- tive Schwarz preconditioner for training scientific machine learning applications.arXiv preprint arXiv:2406.10997, 2024

    Youngkyu Lee, Alena Kopaničáková, and George Em Karniadakis. Two-level overlapping addi- tive Schwarz preconditioner for training scientific machine learning applications.arXiv preprint arXiv:2406.10997, 2024. 28

    2024

  45. [54]

    Fast meta-solvers for 3D complex-shape scatterers using neural operators trained on a non-scattering problem.Computer Methods in Applied Mechanics and Engineering, 446:118231, 2025

    Youngkyu Lee, Shanqing Liu, Zongren Zou, Adar Kahana, Eli Turkel, Rishikesh Ranade, Jay Pathak, and George Em Karniadakis. Fast meta-solvers for 3D complex-shape scatterers using neural operators trained on a non-scattering problem.Computer Methods in Applied Mechanics and Eng...

    2025

  46. [55]

    Multigrid-augmented deep learning precondition- ers for the Helmholtz equation using compact implicit layers

    Bar Lerer, Ido Ben-Yair, and Eran Treister. Multigrid-augmented deep learning precondition- ers for the Helmholtz equation using compact implicit layers. SIAM Journal on Scientific Computing, 46(5):S123–S144, 2024

    2024

  47. [56]

    D3M: A deep domain decomposition method for partial differential equations.IEEE Access, 8:5283–5294, 2019

    Ke Li, Kejun Tang, Tianfan Wu, and Qifeng Liao. D3M: A deep domain decomposition method for partial differential equations.IEEE Access, 8:5283–5294, 2019

    2019

  48. [57]

    Deep domain decomposition method: El- liptic problems

    Wuyang Li, Xueshuang Xiang, and Yingxiang Xu. Deep domain decomposition method: El- liptic problems. In Mathematical and Scientific Machine Learning, pages 269–286. PMLR, 2020

    2020

  49. [58]

    Learning preconditioners for conjugate gradient pde solvers

    Yichen Li, Peter Yichen Chen, Tao Du, and Wojciech Matusik. Learning preconditioners for conjugate gradient pde solvers. InInternational Conference on Machine Learning, pages 19425–19439. PMLR, 2023

    2023

  50. [59]

    Fourier neural operator for parametric partial differential equations

    Zongyi Li, Nikola Borislavov Kovachki, Kamyar Azizzadenesheli, Burigede liu, Kaushik Bhat- tacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations. InInternational Conference on Learning Representations, 2021

    2021

  51. [60]

    Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators

    Lu Lu, Pengzhan Jin, Guofei Pang, Zhongqiang Zhang, and George Em Karniadakis. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence, 3(3):218–229, 2021

    2021

  52. [61]

    A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data.Computer Methods in Applied Mechanics and Engineering, 393:114778, 2022

    Lu Lu, Xuhui Meng, Shengze Cai, Zhiping Mao, Somdatta Goswami, Zhongqiang Zhang, and George Em Karniadakis. A comprehensive and fair comparison of two neural operators (with practical extensions) based on fair data.Computer Methods in Applied Mechanics and Engineering, 393:114...

    2022

  53. [62]

    Accelerating GMRES with deep learning in real-time.arXiv preprint arXiv:2103.10975, 2021

    Kevin Luna, Katherine Klymko, and Johannes P Blaschke. Accelerating GMRES with deep learning in real-time.arXiv preprint arXiv:2103.10975, 2021

    2021 arXiv

  54. [63]

    Transolver++: An accurate neural solver for PDEs on million-scale geometries

    HuakunLuo, HaixuWu, HangZhou, LanxiangXing, YichenDi, JianminWang, andMingsheng Long. Transolver++: An accurate neural solver for PDEs on million-scale geometries. InForty- second International Conference on Machine Learning, 2025

    2025

  55. [64]

    Learning algebraic multigrid using graph neural networks

    Ilay Luz, Meirav Galun, Haggai Maron, Ronen Basri, and Irad Yavneh. Learning algebraic multigrid using graph neural networks. In International Conference on Machine Learning, pages 6489–6499. PMLR, 2020

    2020

  56. [65]

    Marek, D.B

    I. Marek, D.B. Szyld, and M. VohralÃk. Deflation techniques for conjugate gradient methods. Numerical Linear Algebra with Applications, 2(2):155–168, 1995

    1995

  57. [66]

    A neural network multigridsolverfortheNavier-Stokesequations

    Nils Margenberg, Dirk Hartmann, Christian Lessig, and Thomas Richter. A neural network multigridsolverfortheNavier-Stokesequations. Journal of Computational Physics, 460:110983, 2022

    2022

  58. [67]

    SIAM, Philadelphia, 2006

    Gérard Meurant.The Lanczos and conjugate gradient algorithms: From theory to finite preci- sion computations. SIAM, Philadelphia, 2006. 29

    2006

  59. [68]

    Machine-learning-based spectral methods for partial differential equations.Scientific Reports, 13(1):1739, 2023

    Brek Meuris, Saad Qadeer, and Panos Stinis. Machine-learning-based spectral methods for partial differential equations.Scientific Reports, 13(1):1739, 2023

    2023

  60. [69]

    Lighthouse: an automated solver selection tool

    Pate Motter, Kanika Sood, Elizabeth Jessup, and Boyana Norris. Lighthouse: an automated solver selection tool. InProceedings of the 3rd International Workshop on Software Engineering for High Performance Computing in Computational Science and Engineering, pages 16–24, 2015

    2015

  61. [70]

    Deflation of conjugate gradients with applications to boundary value prob- lems

    Roy A Nicolaides. Deflation of conjugate gradients with applications to boundary value prob- lems. SIAM Journal on Numerical Analysis, 24(2):355–365, 1987

    1987

  62. [71]

    Graph neural network-based preconditioners for optimizing gmres algorithm, 2024

    Aleix Nieto Juscafresa. Graph neural network-based preconditioners for optimizing gmres algorithm, 2024

    2024

  63. [72]

    Acceler- ating hypersonic reentry simulations using deep learning-based hybridization (with guarantees)

    Paul Novello, Gaël Poëtte, David Lugato, Simon Peluchon, and Pietro Marco Congedo. Acceler- ating hypersonic reentry simulations using deep learning-based hybridization (with guarantees). Journal of Computational Physics, 498:112700, 2024

    2024

  64. [73]

    Recy- cling Krylov subspaces for sequences of linear systems.SIAM Journal on Scientific Computing, 28(5):1651–1674, 2006

    Michael L Parks, Eric De Sturler, Greg Mackey, Duane D Johnson, and Spandan Maiti. Recy- cling Krylov subspaces for sequences of linear systems.SIAM Journal on Scientific Computing, 28(5):1651–1674, 2006

    2006

  65. [74]

    PyTorch: An impera- tive style, high-performance deep learning library.Advances in neural information processing systems, 32, 2019

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. PyTorch: An impera- tive style, high-performance deep learning library.Advances in neural information processing systems, 32, 2019

    2019

  66. [75]

    Firedrake: Automating the finite element method by composing abstractions

    Florian Rathgeber, David A Ham, Lawrence Mitchell, Michael Lange, Fabio Luporini, An- drew TT McRae, Gheorghe-Teodor Bercea, Graham R Markall, and Paul HJ Kelly. Firedrake: Automating the finite element method by composing abstractions. ACM Transactions on Mathematical Softwar...

    2016

  67. [76]

    Technische Universität Dortmund, Fakultät für Mathematik, Dortmund, 2018

    Hannes Ruelmann, Markus Geveler, and Stefan Turek.On the prospects of using machine learn- ing for the numerical simulation of PDEs: training neural networks to assemble approximate inverses. Technische Universität Dortmund, Fakultät für Mathematik, Dortmund, 2018

    2018

  68. [77]

    Analysis of augmented Krylov subspace methods

    Yousef Saad. Analysis of augmented Krylov subspace methods. SIAM Journal on Matrix Analysis and Applications, 18(2):435–449, 1997

    1997

  69. [78]

    SIAM, Philadelphia, 2003

    Yousef Saad.Iterative methods for sparse linear systems. SIAM, Philadelphia, 2003

    2003

  70. [79]

    A deflated version of the conjugate gradient algorithm.SIAM Journal on Scientific Computing, 21(5):1909–1926, 2000

    Yousef Saad, Manshung Yeung, Jocelyne Erhel, and Frédéric Guyomarc’h. A deflated version of the conjugate gradient algorithm.SIAM Journal on Scientific Computing, 21(5):1909–1926, 2000

    1909

  71. [80]

    Abstract robust coarse spaces for systems of PDEs via generalized eigenprob- lems in the overlaps.Numerische Mathematik, 126:741–770, 2014

    Nicole Spillane, Victorita Dolean, Patrice Hauret, Frédéric Nataf, Clemens Pechstein, and Robert Scheichl. Abstract robust coarse spaces for systems of PDEs via generalized eigenprob- lems in the overlaps.Numerische Mathematik, 126:741–770, 2014

    2014

  72. [81]

    PhD thesis, University of Houston, 2020

    Rita Stanaityte.ILU and Machine Learning Based Preconditioning for the Discretized Incom- pressible Navier-Stokes Equations. PhD thesis, University of Houston, 2020. 30

    2020

  73. [82]

    Optimization-based algebraic multigrid coarsening using reinforcement learning.Advances in neural information processing systems, 34:12129–12140, 2021

    Ali Taghibakhshi, Scott MacLachlan, Luke Olson, and Matthew West. Optimization-based algebraic multigrid coarsening using reinforcement learning.Advances in neural information processing systems, 34:12129–12140, 2021

    2021

  74. [83]

    MG-GNN: Multigrid graph neural networks for learning multilevel domain de- composition methods

    Ali Taghibakhshi, Nicolas Nytko, Tareq Uz Zaman, Scott Maclachlan, Luke Olson, and Matthew West. MG-GNN: Multigrid graph neural networks for learning multilevel domain de- composition methods. InProceedings of the 40th International Conference on Machine Learn- ing, volume 202...

    2023

  75. [84]

    Accelerating Eulerian fluid simulation with convolutional networks

    Jonathan Tompson, Kristofer Schlachter, Pablo Sprechmann, and Ken Perlin. Accelerating Eulerian fluid simulation with convolutional networks. InInternational Conference on Machine Learning, pages 3424–3433. PMLR, 2017

    2017

  76. [85]

    Solver-in-the- loop: Learning from differentiable physics to interact with iterative PDE-solvers.Advances in Neural Information Processing Systems, 33:6111–6122, 2020

    Kiwon Um, Robert Brand, Yun Raymond Fei, Philipp Holl, and Nils Thuerey. Solver-in-the- loop: Learning from differentiable physics to interact with iterative PDE-solvers.Advances in Neural Information Processing Systems, 33:6111–6122, 2020

    2020

  77. [86]

    Henk A Van der Vorst. An iterative solution method for solving f (A) x= b, using Krylov subspace information obtained for the symmetric positive definite matrix A.Journal of Com- putational and Applied Mathematics, 18(2):249–263, 1987

    1987

  78. [87]

    Two-level algebraic multigrid for the Helmholtz problem

    Petr Vanek, Jan Mandel, and Marian Brezina. Two-level algebraic multigrid for the Helmholtz problem. Contemporary Mathematics, 218(349-356):187, 1998

    1998

  79. [88]

    PhD thesis, Inria Bordeaux-Sud Ouest, 2021

    Nicolas Venkovic, Paul Mycek, Luc Giraud, and Olivier Le Maitre.Recycling Krylov subspace strategies for sequences of sampled stochastic elliptic equations. PhD thesis, Inria Bordeaux-Sud Ouest, 2021

    2021

  80. [89]

    C. Vuik, A. Segal, and J. A. Meijerink. A comparison of deflation and coarse grid correction applied to porous media flow.SIAM Journal on Scientific Computing, 20(6):2036–2056, 1999

    1999

  81. [90]

    Learning-based local weighted least squares for algebraic multigrid method.Journal of Computational Physics, page 112437, 2023

    Fan Wang, Xiang Gu, Jian Sun, and Zongben Xu. Learning-based local weighted least squares for algebraic multigrid method.Journal of Computational Physics, page 112437, 2023

    2023

  82. [91]

    Data-driven multi-grid solver for accelerated pressure projection.Com- puters & Fluids, 246:105620, 2022

    Gabriel D Weymouth. Data-driven multi-grid solver for accelerated pressure projection.Com- puters & Fluids, 246:105620, 2022

    2022

  83. [92]

    An additive variant of the Schwarz alternating method for the case of many subregions

    Olof Widlund and Maksymilian Dryja. An additive variant of the Schwarz alternating method for the case of many subregions. 1987

    1987

  84. [93]

    Transolver: A fast transformer solver for PDEs on general geometries

    Haixu Wu, Huakun Luo, Haowen Wang, Jianmin Wang, and Mingsheng Long. Transolver: A fast transformer solver for PDEs on general geometries. InProceedings of the 41st International Conference on Machine Learning, volume 235 ofProceedings of Machine Learning Research, pages 53681...

    2024

  85. [94]

    Blending neural operators and relaxation methods in PDE nu- merical solvers.Nature Machine Intelligence, 6(11):1303–1313, 2024

    Enrui Zhang, Adar Kahana, Alena Kopaničáková, Eli Turkel, Rishikesh Ranade, Jay Pathak, and George Em Karniadakis. Blending neural operators and relaxation methods in PDE nu- merical solvers.Nature Machine Intelligence, 6(11):1303–1313, 2024. 31

    2024

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.