REVIEW 3 major objections 5 minor 30 references
Neural operator preconditioning from mixed dataset for the Helmholtz equations: Application to transcranial ultrasound
T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read A 64×64-trained U-Net preconditioner solves 512×512 skull-ultrasound Helmholtz problems in minutes, where plain GMRES takes hours or stalls.
desk verdict A genuine, honestly-reported extension of the author's own learned-preconditioning framework; the headline 64x64-to-512x512 head-CT result is worth taking seriously, but the grid-transfer premise rests on only four CT slices and deserves scrutiny before the strong generalization claim is accepted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the U-Net as a learned approximate inverse of the discrete Helmholtz operator, trained with a residual (backward-error) loss so that A(c)Nθ(c,σ)≈I. Its 5-channel input encodes real and imaginary parts of the source, the velocity field, and the real and imaginary parts of the PML damping; the 2-channel output is the predicted complex wavefield. Inside FGMRES, the preconditioner application z_j = Nθ(v_j, c, σ) is computed in 32-bit precision while the rest of the iteration runs in 64-bit, making the preconditioner nonlinear yet still valid within the flexible Arnoldi relation. Because the network is convolutional and applied matrix-free, no Helmholtz matrix assemb
What would settle it
Apply the D0-trained U-Net as an FGMRES preconditioner to ten held-out adult head-CT slices at 512×512 with the same focused-arc source; if any case fails to reach η=10^-3 within 5120 iterations, or if the same anatomy discretized at 256, 512, and 1024 shows iteration counts growing steeply with resolution, the claimed coarse-to-fine transfer and strong out-of-distribution generalization would be contradicted.
Extended reading notes
Core claim
The central discovery is that a convolutional U-Net, trained only on 64×64 grids to minimize the normalized residual ||b − A(c)Nθ(b,c,σ)||/||b||, transfers to 512×512 operators with realistic skull velocity fields and still gives FGMRES a useful search direction. Used as a nonlinear right preconditioner in FGMRES—each Krylov vector is passed through the network in 32-bit arithmetic—the method solves all four tested head-CT transcranial ultrasound problems to 10^-3, and to 10^-6 when the network is trained on the mixed D0 dataset, while GMRES converges on only one problem within 5120 iterations and the learned optimizer stagnates. The paper attributes this to the mixed training datasets: idea
Load-bearing premise
A U-Net whose weights were fitted to approximate the inverse Helmholtz operator on 64×64 grids remains a useful preconditioner when the same fixed-receptive-field network is applied to 512×512 operators—64 times more unknowns—with the paper's evidence for this transfer resting on four head-CT slices.
Editorial extensions
If this is right
- One coarse-trained network can accelerate a whole family of parametric Helmholtz solves across resolutions, removing the need to retrain per mesh.
- Practical 2D transcranial ultrasound planning becomes computationally feasible on CPUs: four head-CT problems to 10^-3 in about 12 minutes and to 10^-6 in about 46 minutes.
- The matrix-free character sidesteps the memory and setup costs of classical algebraic preconditioners (Jacobi, incomplete LU, algebraic multigrid) on fine grids.
- Because FGMRES corrects the network's errors, the final solution accuracy is not capped by the network's own accuracy; the hybrid can reach arbitrarily tight tolerances.
- The mixed-dataset result identifies dataset composition as a controllable factor: D0 outperforms both the purely specialized and the purely randomized training sets on the hardest out-of-distribution cases.
Reading between the lines
- If the 64→512 transfer is real, a natural stress test is to apply the same D0 network at 256, 512, and 1024 resolutions on identical CT slices; the resolution at which iteration counts degrade would map the effective range of the fixed receptive field.
- The same recipe should extend to 3D transcranial volumes and higher frequencies, but the paper only demonstrates 2D and 490 kHz; those settings remain open.
- Dataset balance looks like a tunable hyperparameter: interpolating between D0 and D1 could let a user optimize for a specific anatomy or transducer geometry without changing the architecture.
- The comparison against a standalone learned optimizer suggests the Krylov wrapper carries much of the robustness; the same network that stagnates as a solver becomes reliable as a preconditioner, pointing toward hybrid designs as a general pattern.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a learned nonlinear preconditioner for FGMRES solving sequences of 2D Helmholtz equations in transcranial ultrasound. A depth-4 U-Net is trained on 64×64 grids with a relative-residual (physics-informed) loss using six mixed datasets (D0–D5) combining idealized skulls, random velocity fields, and several source distributions. The network is then used as a matrix-free right preconditioner in FGMRES. The central empirical claim is in Section 4.2.2 (Table 8): the D0-trained preconditioner solves four 512×512 head-CT problems to η_b = 10^{-3} in at most 191 iterations (~12 min CPU total) and to 10^{-6} in ~46 min, whereas unpreconditioned GMRES converges on only one of the four within 5120 iterations (~12.5 h) and the learned optimizer of [26] stagnates/diverges on three of the four. The paper also reports speedups on 64×64 in-distribution tests and on idealized-skull/seismology-style benchmarks, and concludes that mixed dataset design improves generalization. The relative-residual loss and the stopping criterion are coherent; the main risk identified is the unanalyzed transfer from 64×64 training to 512×512 inference.
Significance. If the results hold, the practical significance is real: a matrix-free learned preconditioner that brings FGMRES to 512×512 transcranial ultrasound problems, with honest per-slice iteration counts, wall-clock times, same-hardware comparisons, and an unsupervised physics-informed loss. The paper is incremental over the authors' earlier framework [9]; the contribution is dataset design rather than architecture. Its strengths are the systematic six-dataset comparison, the 10^{-3}/10^{-6} head-CT results, and the clearly reported failure cases (e.g., D5 on one CT slice). The significance is limited by the four-slice real-data evaluation, the lack of resolution-transfer analysis, and some overclaimed generalization statements, but the evidence is consistent and not internally contradictory.
major comments (3)
- [Section 4.2.2, Table 8] The transfer from 64×64 training to 512×512 inference is the load-bearing premise, but it is only asserted ('this computational grid is 64 times larger than the 64×64 grid used during training'). The depth-4 U-Net with stride-2 downsampling maps 512×512 inputs to a 32×32 bottleneck, a resolution never seen in training; the convolutional kernels and batch-normalization statistics were not adapted. No intermediate-resolution experiments (128, 256) or diagnostics of N_θ's residual reduction on fine grids are given. Since the central practical claim depends on this transfer, please add such a resolution study or, at minimum, a larger CT cohort with per-slice results and a statement of slice selection. Without this, the four-slice evidence is not sufficient to establish the claimed out-of-distribution generalization.
- [Table 8 / Figure 9(c)] The claim of 'strong out-of-distribution generalization' is supported by only four CT slices, with no selection criteria or statistics. The D5-trained preconditioner fails on one of the four (5120*), and iteration counts vary by factors of 2–10 across D0–D5 on the same four problems. With n=4, the ranking of datasets could easily be slice-specific. Please provide a larger, systematically sampled test set, report per-example costs, or qualify the generalization claim to the four reported examples. This is load-bearing because the abstract and conclusions make a broad generalization claim.
- [Abstract and Section 4.2.2] The statement that the method 'achieves arbitrary solution accuracies' is stronger than the evidence. Table 8 reports η_b = 10^{-3} and 10^{-6} for the 512×512 head-CT problems; no 10^{-9} or 10^{-12} result is provided at that scale, and the learned optimizer comparison is only made at 10^{-3}. Please either add a tighter-tolerance head-CT run or rephrase the claim as 'high prescribed accuracies (demonstrated to 10^{-6})'. This is a claim-calibration issue rather than a technical error.
minor comments (5)
- [Section 2.1 vs Section 4.2.2] The paper restricts the Helmholtz formulation to ω=1, but Section 4.2.2 describes the CT problem at 490 kHz. Please clarify the nondimensionalization/scaling; as written, the frequency statement is unused and confusing.
- [Tables 6–9] The star notation is inconsistent. In (Its_max, Its_avg, Its_min)* the star denotes divergence, but a superscript star after dataset labels (e.g., D4⋆ in Table 7, D0⋆ in Table 8, D3⋆ in Table 9) is never explained.
- [Figure 3] The caption says '8 × 8 mixed speed of sounds', but the axes and text indicate 64×64 training samples. Please correct this typo.
- [Section 3.1, Eq. (2), Eq. (8)] Equation (2) defines σ as a real damping function, but Section 3.1 and Eq. (8) describe σ as complex. Please reconcile the notation.
- [Reproducibility] No data or code availability statement is provided. Given the paper's empirical nature, releasing the CT preprocessing script, trained checkpoints, and dataset-generation code would materially aid reproducibility.
Circularity Check
No circular steps: training objective, FGMRES stopping criterion, and test data are distinct; self-citations are contextual only.
full rationale
The paper's chain is: train N_theta to minimize the relative residual of the discrete Helmholtz equation on 64x64 systems (Eq. 11); insert N_theta as a nonlinear preconditioner into FGMRES (Algorithm 1, Eq. 7); measure the number of FGMRES iterations/elapsed time needed to drive the same relative residual eta_b below epsilon (Eq. 12) on 64x64 test problems, idealized skulls, and 512x512 head-CT problems. The use of the relative residual in both the training loss and the stopping criterion is a coherent choice of metric, not a fitted quantity renamed as a prediction: the reported convergence is obtained by actually running FGMRES, not by reading off the training loss. All six preconditioners are trained on source/velocity distributions that do not include the head-CT data (qure.ai, [22]) or the seismology/spherical-wave tests, and the learned-optimizer baseline is an external checkpoint from [26]; the decisive comparisons are against external or held-out objects. The self-citations [9,25,29,30] supply the preconditioning framework and prior comparisons, but the numerical evidence in Tables 6-9 is generated in this paper and does not reduce to the contents of those citations. The main caveats—the unproven transfer from 64x64 training to 512x512 application and the post-hoc identification of D0 as best on the four CT slices—are empirical generalization/selection concerns, not circularity by construction. No equation in the paper makes the test-time residual equal to the training loss by definition, and no load-bearing claim is justified solely by a self-citation.
Assumptions & free parameters
free parameters (5)
- training-dataset mixture probabilities =
c: [0.4,0.4,0.1,0.1]; b: [0.6,0.3,0.05,0.05]
- dirac source amplitude =
60 (dirac/dirac1), 30 (dirac4)
- U-Net hyperparameters =
depth 4, 32 channels, kernel 8, stride 2, batch 32, LR 1e-3 to 1e-5, 500 epochs
- trained network weights theta =
~832K parameters per dataset (final checkpoints not released)
- PML and frequency settings =
l=4, sigma_max=2, omega=1
assumptions (5)
- domain assumption Fourier spectral discretization with a PML is an adequate representation of the unbounded-domain Helmholtz problem
- ad hoc to paper A U-Net trained at 64x64 provides a useful approximate inverse of the Helmholtz operator at 512x512
- domain assumption Truncated idealized skull fields are a sufficient training proxy for real head-CT velocity fields
- standard math FGMRES with a nonlinear preconditioner remains well-defined and convergent as long as the preconditioned basis Z_k is full rank
- domain assumption The 2D Helmholtz model at omega=1 with c in [1,2] captures the practical transcranial ultrasound regime
Cite this review
Pith. "Pith review of Neural operator preconditioning from mixed dataset for the Helmholtz equations: Application to transcranial ultrasound." pith.science (2026). https://pith.science/paper/XHZW6YV7
@misc{pith2026260716906,
author = {Pith},
title = {Pith review of: Neural operator preconditioning from mixed dataset for the Helmholtz equations: Application to transcranial ultrasound},
year = {2026},
howpublished = {\url{https://pith.science/paper/XHZW6YV7}},
note = {Machine review of arXiv:2607.16906}
}
read the original abstract
This work develops a neural operator preconditioned subspace method for sequences of linear systems arising from the discretization of the two-dimensional Helmholtz equation in transcranial ultrasound applications. The problem involves strongly heterogeneous, patient-dependent velocity fields that induce severe wave distortion and pose significant challenges for standard iterative solvers. Building on neural network preconditioning framework of Giraud et al. (HAL RR-9593, 2025) and the idealized skull dataset used for the learned optimizer of Stanziola et al. (JCP 441, 2021), neural operator preconditioners are trained on six mixed velocity-source datasets combining randomized source configurations and idealized skull-based velocity fields with random noise. The proposed mixed-dataset strategy aims to improve both computational efficiency and generalization across varying configurations. The neural operator is trained on a coarse grid using a physics-informed loss based on the relative residual of the discrete Helmholtz equation and is incorporated as a nonlinear preconditioner within flexible GMRES (FGMRES). Numerical experiments demonstrate that the resulting hybrid method efficiently solves practical transcranial ultrasound problems on grids 64 times larger than those used during training, whereas both classical GMRES and the learned optimizer fail to converge within comparable computational budgets. Moreover, the proposed method achieves arbitrary solution accuracies and exhibits strong out-of-distribution generalization across diverse source and velocity configurations. This work highlights the importance of dataset design in scientific machine learning and provides a practical framework for integrating matrix-free neural operator preconditioning with Krylov subspace methods for solving practical large-scale Helmholtz problems.
Figures
Figures from the paper (12 more)
Reference graph
Works this paper leans on
-
[26]
A Helmholtz Equation Solver Using Un- supervised Learning: Application to Transcranial Ultrasound
A. Stanziola, S. R. Arridge, B. T. Cox, and B. E. Treeby. “A Helmholtz Equation Solver Using Un- supervised Learning: Application to Transcranial Ultrasound”. In:Journal of Computational Physics 441 (2021), p. 110430.issn: 00219991.doi:10 . 1016 / j . jcp . 2021 . 110430.url:https : / / www . sciencedirect.com/science/article/abs/pii/S0021999121003259
2021
-
[9]
Giraud, C
L. Giraud, C. Kruse, P. Mycek, M. Shpakovych, and Y.-F. Xiang.Neural network preconditioning: a case study for the solution of the parametric Helmholtz equation. Technical Report 9593. Inria Centre at the University of Bordeaux, 2025.url:https://inria.hal.science/hal-05157038
2025
-
[1]
Conditional Denoising Diffusion Model-BasedRobustMRImageReconstructionfromHighlyUndersampledData
M. Alsubaie, W. Liu, L. Gu, O. C. Andronesi, S. M. Perera, and X. Li. “Conditional Denoising Diffusion Model-BasedRobustMRImageReconstructionfromHighlyUndersampledData”.In:arXiv:2510.06335 (2025).doi:10.48550/arXiv.2510.06335. arXiv:2510.06335.url:http://arxiv.org/abs/2510. 06335
-
[2]
Multigrid-Augmented Deep Learning Preconditioners for the Helmholtz Equation
Y. Azulay and E. Treister. “Multigrid-Augmented Deep Learning Preconditioners for the Helmholtz Equation”. In:SIAM Journal on Scientific Computing45.3 (2023), S127–S151.issn: 1064-8275.doi: 10.1137/21M1433514.url:https://epubs.siam.org/doi/10.1137/21M1433514
work page doi:10.1137/21m1433514.url:https://epubs.siam.org/doi/10.1137/21m1433514 2023
-
[3]
A Perfectly Matched Layer for the Absorption of Electromagnetic Waves
J.-P. Berenger. “A Perfectly Matched Layer for the Absorption of Electromagnetic Waves”. In:Journal of Computational Physics114.2 (1994), pp. 185–200.issn: 0021-9991.doi:10.1006/jcph.1994.1159. url:https://www.sciencedirect.com/science/article/pii/S0021999184711594
arXiv 1994
-
[4]
C. M. Bishop and H. Bishop. “Autoencoders”. In:Deep Learning: Foundations and Concepts. Cham: Springer International Publishing, 2024, pp. 563–579.isbn: 978-3-031-45468-4.doi:10.1007/978-3- 031-45468-4_19.url:https://doi.org/10.1007/978-3-031-45468-4_19
doi:10.1007/978-3- 2024
-
[5]
Message Passing Neural PDE Solvers
J. Brandstetter, D. E. Worrall, and M. Welling. “Message Passing Neural PDE Solvers”. In:Interna- tional Conference on Learning Representations (ICLR). 2022.doi:10.48550/arXiv.2202.03376
-
[6]
OpenFWI: Large-scale Multi-structural Benchmark Datasets for Full Waveform Inversion
C.-Y. Deng, S.-H. Feng, H.-C. Wang, X.-T. Zhang, P. Jin, Y.-N. Feng, Q.-L. Zeng, Y.-P. Chen, and Y.-Z. Lin. “OpenFWI: Large-scale Multi-structural Benchmark Datasets for Full Waveform Inversion”. In:Advances in Neural Information Processing Systems. Ed. by S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh. Vol. 35. Curran Associates, Inc.,...
2022
Show all 30 references
-
[7]
Numerical Solution of Mixed-Dimensional PDEs Using a Neural Preconditioner
N. Dimola, N. R. Franco, and P. Zunino. “Numerical Solution of Mixed-Dimensional PDEs Using a Neural Preconditioner”. In: arXiv:2505.08491 (2025).doi:10 . 48550 / arXiv . 2505 . 08491.url: https://arxiv.org/pdf/2505.08491
-
[8]
Scientific machine learning
F. Dietrich and W. Schilders. “Scientific machine learning”. In:Math Semesterber72 (2025), pp. 89– 115.doi:10.1007/s00591-025-00399-4
2025 doi
-
[10]
N.J.Higham.Accuracy and stability of numerical algorithms.SIAM,2002.doi:10.1137/1.9780898718027
2002 doi
-
[11]
DeepONet Based Preconditioning Strategies for Solving Para- metric Linear Systems of Equations
A. Kopaničáková and G. Karniadakis. “DeepONet Based Preconditioning Strategies for Solving Para- metric Linear Systems of Equations”. In:SIAM Journal on Scientific Computing47.1 (2025), pp. C151– C181.issn: 1095-7197.doi:10.1137/24m162861x.url:http://dx.doi.org/10.1137/24M162861X
2025 doi
-
[12]
Neural Operator: Learning Maps Between Function Spaces with Applications to PDEs
N. B. Kovachki, Z. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. M. Stuart, and A. Anandkumar. “Neural Operator: Learning Maps Between Function Spaces with Applications to PDEs”. In:Journal of Machine Learning Research24.89 (2023), pp. 1–97.url:http : / / jmlr . org / pa...
2023
- [13]
-
[14]
Fourier Neural Operator for Parametric Partial Differential Equations
Z. Li, N. B. Kovachki, K. Azizzadenesheli, B. liu, K. Bhattacharya, A. Stuart, and A. Anandkumar. “Fourier Neural Operator for Parametric Partial Differential Equations”. In:International Conference on Learning Representations. 2021.url:https://openreview.net/forum?id=c8P9NQVtmnO
2021
-
[15]
AI-Augmented Adaptive Digital Twin Modeling for Brain Tumor Evolution Prediction and Treatment Scheduling
W. Liu, M. Trimboli, and X. Li. “AI-Augmented Adaptive Digital Twin Modeling for Brain Tumor Evolution Prediction and Treatment Scheduling”. In: arXiv:2607.13877 (2026).doi:10.48550/arXiv. 2607.13877. arXiv:2607.13877.url:http://arxiv.org/abs/2607.13877. 22
-
[16]
Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators
L. Lu, P. Jin, G. Pang, Z. Zhang, and G. Karniadakis. “Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators”. In:Nature Machine Intelligence3 (2021), pp. 218–229.doi:10.1038/s42256-021-00302-5
2021 doi
-
[17]
Neural Operator: Learning Maps Between Function Spaces With Applications to PDEs
N. Kovachki, Z.-Y. Li, B. Liu, K. Azizzadenesheli, K. Bhattacharya, A. Stuart, and A. Anandkumar. “Neural Operator: Learning Maps Between Function Spaces With Applications to PDEs”. In:Journal of Machine Learning Research(2023). http://jmlr.org/papers/v24/21-1524.html, pp. 1–97
2023
-
[18]
Pathak, S
J. Pathak, S. Subramanian, P. Harrington, S. Raja, A. Chattopadhyay, M. Mardani, T. Kurth, D. Hall, Z. Li, K. Azizzadenesheli, P. Hassanzadeh, K. Kashinath, and A. Anandkumar.FourCastNet: A Global Data-driven High-resolution Weather Model using Adaptive Fourier Neural Operator...
2022 arXiv
-
[19]
Meta-learning PINN loss functions
A. F. Psaros, K. Kawaguchi, and G. E. Karniadakis. “Meta-learning PINN loss functions”. In:Jour- nal of Computational Physics458 (2022), p. 111121.issn: 0021-9991.doi:https://doi.org/10. 1016 / j . jcp . 2022 . 111121.url:https : / / www . sciencedirect . com / science / artic...
2022
-
[20]
On the Compatibility of a Given Solution With the Data of a Linear System
J. L. Rigal and J. Gaches. “On the Compatibility of a Given Solution With the Data of a Linear System”. In:Journal of the ACM14.3 (1967), pp. 543–548.doi:10 . 1145 / 321406 . 321416.url: https://doi.org/10.1145/321406.321416
1967
-
[21]
U-Net: Convolutional Networks for Biomedical Image Seg- mentation
O. Ronneberger, P. Fischer, and T. Brox. “U-Net: Convolutional Networks for Biomedical Image Seg- mentation”. In: arXiv:1505.04597 (2015).doi:10.48550/arXiv.1505.04597. arXiv:1505.04597.url: http://arxiv.org/abs/1505.04597
-
[22]
Development and validation of deep learning algorithms for detection of critical findings in head CT scans
S. Chilamkurthy, R. Ghosh, S. Tanamala, M. Biviji, N. G. Campeau, V. K. Venugopal, V. Mahajan, P. Rao, and P. Warier. “Development and validation of deep learning algorithms for detection of critical findings in head CT scans”. In: arXiv:1803.05854 (2018).doi:10.48550/arXiv.18...
-
[23]
A flexible inner-outer preconditioned GMRES algorithm
Y. Saad. “A flexible inner-outer preconditioned GMRES algorithm”. In:SIAM Journal Scientific Com- puting14 (1993), pp. 461–469
1993
-
[24]
GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems
Y. Saad and M. Schultz. “GMRES: a generalized minimal residual algorithm for solving nonsymmetric linear systems.” In:SIAM Journal on Scientific and Statistical Computing7 (1986), pp. 856–869
1986
-
[25]
Shpakovych.Neural Network Preconditioning of Large Linear Systems
M. Shpakovych.Neural Network Preconditioning of Large Linear Systems. Technical Report 0518. Inria Centre at the University of Bordeaux, 2023.url:https://inria.hal.science/hal-04254315
2023
-
[27]
Efficient preconditioning for iterative methods with graph neural networks
V. Trifonov, A. Rudikov, O. Iliev, Y. M. Laevsky, and I. Oseledets. “Efficient preconditioning for iterative methods with graph neural networks”. In:AI4X 2025 International Conference. 2025.url: https://openreview.net/pdf?id=PjDJonBo9T
2025
-
[28]
J. H. Wilkinson.Rounding Errors in Algebraic Processes. Notes on Applied Science 32. Also published by Prentice-Hall, Englewood Cliffs, NJ, USA, 1964. Reprinted by Dover Publications, New York, 1994. London, UK: HMSO, 1963
1964
-
[29]
Solution of Large Linear Systems with a Massive Number of Right-Hand Sides and Ma- chine Learning
Y.-F. Xiang. “Solution of Large Linear Systems with a Massive Number of Right-Hand Sides and Ma- chine Learning”. PhD thesis. University of Bordeaux, 2022.url:https://theses.fr/2022BORD0383
2022
-
[30]
Xiang.Unsupervised convolution neural operator preconditioning for the solution of some hetero- geneous fluid PDEs
Y.-F. Xiang.Unsupervised convolution neural operator preconditioning for the solution of some hetero- geneous fluid PDEs. Technical Report. Inria Centre at the University of Bordeaux and Inria Center of the University of Lorraine, 2025.url:https://inria.hal.science/hal-0488693...
2025
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.