{"id":"d310c83e-41f9-4ea5-94f7-96fe809371b6","arxiv_id":"2411.13341","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Masked self-attention DeepONet preconditioners trained on simple domains help HINTS converge on Helmholtz problems over unrelated geometries, particularly when the domain has cracks or holes.","lead":"This paper introduces a geometry-aware neural preconditioner that can be trained on simple shapes and still help solve the Helmholtz equation on quite different geometries without retraining. The practical read is that learned iterative solvers may become usable for engineering domains involving cracks, holes, or obstacles, where standard extensions of vanilla DeepONet fail to converge.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The geometry-aware mechanism is defined only on the fixed 15x15 training grid, so the headline transferability claim is unproven for narrow cracks or holes that fall between training points.","rationale":"The paper's genuinely novel component is the masked 225-point geometry representation, and the numerical evidence for the central claim (geometries 1, 2, 6, 7, 9) is consistent with the mask resolving the cracks on the 15x15 training grid. But there is no mechanism by which finer geometric detail can influence N_masked: the mask is built only from the same fixed 225 points used for the branch input, and Remark 1.1 explicitly discusses interpolation of f, not of the geometry descriptor. Since Section 1.2 emphasizes that masking matters precisely when narrow cracks appear, the width/alignment of the tested cracks relative to h=1/14 is a load-bearing condition that is neither stated nor verified. This is not an internal inconsistency, nor a reason to doubt that the reported experiments were run as described; it is a boundary on the domain of the claimed generalization. The proposed experiment directly tests that boundary by placing the distinguishing geometry feature below training-grid resolution. If N_masked still outperforms N_non-masked, the concern is settled in the paper's favor; if not, the abstract and Section 3.1 overstate the transferability. The reader's CONDITIONAL verdict already captures the need for this kind of check, so I recommend no change to the verdict.","tokens_in":12293,"tokens_out":9020,"duration_ms":114885,"concrete_test":"Construct a test domain identical to Polygon 1 (or an obstacle domain) but insert a crack of width less than 1/14 strictly between the 15x15 training-grid lines, keeping all other geometry unchanged; run H-GS(J=40) and H-GMRES(m=30,J=15) with N_masked, N_non-masked, and vanilla. Also rerun the original resolved-crack geometry as a positive control. If N_masked's iteration counts become statistically indistinguishable from N_non-masked, or diverge where the resolved-crack version converges, then the reported geometry-awareness advantage holds only for features resolved by the 225-point mask, and the abstraction's unrestricted transferability claim would need to be weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that N_masked is a robust preconditioner on unseen geometries, with the decisive evidence being 'abrupt changes (cracks/bumps/holes)' in Section 3.1. However, every geometry-aware input—the signed-distance vector O^Ω, the sampled function f^Ω, and the mask M^Ω—is defined on the same fixed 15x15 grid used in training (Section 2.1). The network therefore has no mechanism to see any geometric feature at a finer scale: a crack or hole is encoded only by which of the 225 fixed points lie inside or outside the domain. A feature located strictly between training points is invisible to N_masked, and the masking degenerates to the same zero-extension input used by N_non-masked. Section 3.1's narrow-crack results thus depend on the tested cracks being wide enough, or aligned enough, to be resolved by h=1/14; this condition is never stated or tested. Remark 1.1 addresses only interpolation of the input function f, not refinement of the geometry representation. Moreover, for the h=1/57 test grid, none of the interior training-grid points coincide with test-grid points (gcd(14,57)=1), so the mechanics of applying the fixed-grid branch to that test residual are also unspecified. Without this resolution condition, the evidence supports geometry awareness at training-grid resolution only, not the abstract's unrestricted transferability claim.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a geometry-aware DeepONet preconditioner that combines masked self-attention with the HINTS iterative framework for solving Helmholtz-type linear systems. The network is trained on a fixed 15x15 grid over [0,1]^2 and evaluated on different geometries without fine-tuning, using Richardson iterations hybridized with Gauss-Seidel or GMRES. The main reported evidence is a set of iteration-count tables on 12 polygonal domains and several obstacle/exterior domains, where the masked model converges in cases where vanilla DeepONet or classical iterations diverge.","tokens_in":12564,"tokens_out":4225,"duration_ms":48470,"significance":"If the transferability claim is supported, the architecture would be a useful addition to the toolkit of learned preconditioners, especially for users who cannot rely on convolutional grids. The paper has the merit of evaluating on out-of-distribution geometries against independent baselines (vanilla DeepONet, GMRES, a CNN-based geometry-aware variant) and makes code available. However, the significance is currently limited by the fixed-grid geometry representation, the absence of a protocol for the user-selected skip factor, and the lack of statistical and timing evidence. The contribution is empirical and would become substantially stronger with controlled experiments.","major_comments":[{"comment":"The geometry-aware inputs O^Ω, f^Ω, and M^Ω are all defined on the same fixed 15x15 grid used in training. Geometric features that fall strictly between the 225 grid points, such as narrow cracks or small holes, are therefore invisible to the network, and the mask degenerates to the same zero-extension input used by N_non-masked. Section 3.1 claims that N_masked performs best for geometries with abrupt changes (cracks/bumps/holes), but this can only hold if the tested features are resolved by h=1/14. No such resolution condition is stated or tested. Remark 1.1 addresses interpolation of the input function f, but not refinement of the geometry representation. For the h=1/57 and h=1/224 test grids, none of the training grid points coincide with test grid points, so the mechanism by which the fixed-grid branch is applied to a test residual is unspecified. The paper should either demonstrate that the reported narrow-crack results are insensitive to the fixed sampling grid or restrict the transferability claim to features resolvable on the training grid.","section":"Section 2.1 and Section 3.1"},{"comment":"The method depends critically on the user-selected skip factor J and preconditioner scale α, but no protocol for choosing these parameters is given. The tables use different J values across geometries (e.g., J=40, 60, and 80 in Tables 1-2; J=15 and 100 in Tables 4-5), and the qualitative ranking of methods changes with J: Section 3.1 reports that for the Figure 4 geometries with GMRES, vanilla is better at J=15, whereas at J=5 H-GMRES with vanilla diverges while N_masked converges. Without a fixed hyperparameter protocol or a sensitivity analysis, the comparison across models is not controlled, and the robustness claim is weaker than stated.","section":"Equation (2.3), Tables 1-5"},{"comment":"The paper reports only averaged iteration counts and, with one exception (Table 1, geometry 5: 1620 ± 170), gives no error bars, number of runs, seeds, or per-run statistics. The word \"averaged\" is not sufficient to support a robustness claim. In addition, no wall-clock times are reported despite Remark 3.2 discussing CPU time; iteration count alone is insufficient because the networks have different per-iteration costs. Multi-run statistics and timings should be provided for the main comparisons.","section":"Tables 1-5"},{"comment":"The paper's own tables show that vanilla DeepONet is often faster or similar in iteration count (e.g., Table 1 geometries 3, 4, 5, 8; Table 2 geometries 10, 11, 12; Table 3 geometries 1, 4, 11). The abstract claims that the proposed network 'remains a robust preconditioner' and Section 3.1 states N_masked 'performed better than all methods' for a subset of geometries, but the conditions under which the masked model is preferable are not characterized. The central claim should be scoped precisely: the experiments support an existence claim (there are geometries where N_masked converges when vanilla diverges), not a general superiority claim. A criterion or at least a discussion of when the masked model is expected to help would be needed to support the broader wording.","section":"Section 3.1 and Abstract"}],"minor_comments":[{"comment":"The abstract says 'a simple geometry' but the training data in Figure 1 contain multiple geometries; the plural wording in Section 1.2 is more accurate and should be used consistently.","section":"Abstract and Section 1.2"},{"comment":"Equation (1.7) writes the network update as un+1 = un + D(rn), while (2.3) uses a normalized argument std(rn)/α · N(α rn/std(rn)); the notation should be aligned to avoid confusion about what D is.","section":"Equations (1.7) and (2.3)"},{"comment":"The H-GMRES rows in Tables 4 and 5 do not specify the restart parameter m, unlike Table 3 which states m=30. This makes the results difficult to reproduce and should be corrected.","section":"Tables 4 and 5"},{"comment":"The tables refer to 'geometry 1' through 'geometry 12' but Figure 2 labels the domains as 'Polygon 1' through 'Polygon 12'; the mapping should be made explicit in the text or captions.","section":"Figure 2 and Tables 1-2"},{"comment":"Remark 1.1 states that using grids that contain the training grid is less expensive, but the test grids h=1/57 and h=1/224 do not contain the training grid points; the remark should be reconciled with the actual evaluation procedure.","section":"Remark 1.1"},{"comment":"There is a typo in the title on the first page: 'a re' should be 'are'.","section":"Title"}],"recommendation":"major_revision","confidential_remarks":"The core idea is interesting and the experiments provide some evidence for a narrower claim, but the fixed-grid geometry resolution issue and the uncontrolled hyperparameter selection need to be addressed before the paper can be considered for publication. I would encourage the editor to request a revision that either adds a resolution study for the geometry representation and a fixed hyperparameter protocol, or substantially softens the transferability claim. The manuscript is not ready for acceptance in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Best quick take: the masked self-attention DeepONet is a real new architecture for geometry-aware preconditioning, and the paper earns its main empirical point—on several domains with cracks, bumps, or holes, N_masked converges where vanilla DeepONet with zero extension diverges. The abstract's wording ('robust preconditioner for different geometries without further fine-tuning') is broader than the evidence, because the geometry is only ever seen through a fixed 15x15 grid.\n\nWhat's new: combining masked self-attention inside a DeepONet branch and using it as a HINTS preconditioner, trained on simple shapes and evaluated on unseen ones. The comparison to a naive CNN-based geometry-aware variant (ga-vanilla) is useful—it shows the benefit is not just from feeding a mask. The obstacle and exterior-domain tests extend the story beyond internal cracks. The paper is also honest: it states that vanilla is often faster when it converges, and it says the user should hold both N_masked and N_non-masked.\n\nWhere it's soft: the fixed-grid concern is valid. The branch input—mask, signed distance, sampled f—is defined on the same 225 points used in training. A narrow crack that falls between those points is only visible through small signed-distance values at neighboring points; the mask itself sees nothing. The paper never states what feature resolution it actually requires, so the headline transferability claim is conditional on test features being resolvable at the training grid. Also, the skip factor J is hand-picked per geometry (40-200 in the tables) with no sensitivity study; most iteration counts have no error bars; and there are no wall-clock times. The code link is a bare GitHub URL without a commit hash or dataset, so reproduction may take effort.\n\nOverall: the core mechanism is plausible, the negative result for vanilla on cracked domains is well demonstrated, and the body is more careful than the abstract. The missing piece is a resolution analysis for the geometry encoding. For the learned-preconditioner community, this is worth reading and citing. It deserves a serious referee who will ask for multi-seed statistics, a fixed hyperparameter protocol, and an explicit statement of the geometry-resolution condition.\n\nRecommendation: send it to peer review, with major revision likely on the transferability claim. My own verdict would be conditional acceptance after those revisions.","headline":"A real new architecture with a supported core claim, but the transferability claim needs a resolution caveat.","tokens_in":13092,"tokens_out":5837,"would_cite":true,"duration_ms":63781,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["65F10","65N22","68T07","65N06"],"pacs":[],"model":"deepseek-v4-flash","headline":"Masked self-attention lets a pretrained DeepONet precondition unseen Helmholtz geometries.","keywords":["Helmholtz equation","preconditioner","DeepONet","geometry transferability","masked self-attention","hybrid iterative solver","Gauss-Seidel","GMRES"],"falsifier":"Run HINTS-Gauss-Seidel with $N_{\\mathrm{masked}}$ on a domain containing a crack whose width is less than one training-grid spacing ($1/15$) while the test resolution is $h=1/57$; if the crack falls entirely between the 225 fixed training points, the masking matrix $M^\\Omega$ carries no inside-point information there. The experiment would falsify the paper's claim if the method then behaves like the zero-extension $N_{\\mathrm{non-masked}}$ and fails to converge, since it would show the coarse grid, not the mask, is responsible for geometry transfer.","tokens_in":12056,"feed_emoji":"📐","tokens_out":9440,"duration_ms":89243,"temperature":0.7,"pith_summary":"The paper tries to show that a learned preconditioner for the Helmholtz equation can be trained on a few simple geometries and then solve linear systems on very different geometries without retraining. The payoff would be that expensive geometry-specific data generation is unnecessary: one network, trained on rectangles and simple polygons, can sit inside an iterative solver and make it converge on domains with cracks, bumps, and holes. The paper reports that its masked-attention DeepONet converges in cases where the vanilla DeepONet with zero extension diverges, and that a simple CNN-based geometry-aware variant performs worse. If the claim holds, it would give a cheap and robust preconditioning option for non-symmetric, non-positive-definite PDE problems.","feed_headline":"Masked-attention DeepONet preconditions Helmholtz on unseen geometries","feed_subtitle":"Trained on simple shapes, the net keeps Gauss-Seidel and GMRES converging where the vanilla DeepONet diverges.","key_machinery":"The load-bearing object is the masked attention operation $M$-attention$(v, M^\\Omega) = \\mathrm{softmax}((vv^T + M^\\Omega)/\\sqrt{N})v$, where $M^\\Omega$ is $0$ inside the domain and $-\\infty$ outside. It lets the branch network see only grid points belonging to the current geometry, avoiding any artificial zero extension of $f$ outside $\\Omega$. The branch and trunk networks form a DeepONet-style product, and the whole network is used as the preconditioner $N$ inside the HINTS update: every $J$ classical iterations (Gauss-Seidel or GMRES) are followed by one normalized network correction.","core_discovery":"The paper claims that a DeepONet preconditioner equipped with masked self-attention ($N_{\\mathrm{masked}}$) can be trained only on simple rectangular and polygonal geometries, then used inside the HINTS hybrid iteration to solve Helmholtz systems on different, more complicated geometries without retraining. It reports that on domains with abrupt boundary changes such as cracks, bumps, and holes, $N_{\\mathrm{masked}}$ converges in cases where the vanilla DeepONet with zero extension diverges, and that a naive CNN-based geometry-aware extension usually performs worst. The central discovery is that masking the attention so that points outside the domain are ignored is enough to make geometry transfer work, even though the network input remains a fixed 15-by-15 grid.","pith_inferences":["Editorial extension: the masking idea should transfer to other indefinite or nonsymmetric second-order PDEs, such as convection-diffusion or time-harmonic Maxwell problems, because the architecture never assumes symmetry or definiteness.","The paper leaves open how the method behaves for features finer than the 15-by-15 training grid; a crack that falls entirely between the 225 fixed points may be invisible to the mask, so a natural next test would use a finer training grid or an adaptive point selection.","A practical user currently has to keep both the masked and non-masked networks and choose per geometry, since neither dominates in all tables; unifying or automatically switching between them is a plausible follow-up suggested by the numerical results.","Because the network acts on residuals only every $J$ steps, the experiments suggest that a moderately accurate operator network suffices to steer convergence, which could loosen accuracy requirements for geometry-transfer training in other applications."],"forward_implications":["A HINTS user can train once on simple geometries and deploy the same network as a preconditioner for new geometries, avoiding the cost of generating training data for many domains.","For domains with narrow cracks or holes, the masked model should be chosen over vanilla DeepONet with zero extension, since the mask removes the outside-domain values that mislead the network.","Because the network is non-convolutional and its inputs are point values, the same architecture should apply to non-rectangular grids and point clouds, not only uniform Cartesian grids.","Combined with Gauss-Seidel, the approach keeps a solver that would otherwise diverge converging; combined with GMRES, it reduces both total iterations and the number of expensive network calls.","Each network evaluation costs constant computation because the input dimension is fixed at 225 points, so the hybrid iteration remains $O(N)$ per step for sparse systems."],"supporting_citations":[{"why":"Introduces the HINTS hybrid scheme and the vanilla DeepONet preconditioner baseline that this paper extends.","marker":"[27]"},{"why":"Defines the branch-and-trunk DeepONet structure on which the masked and non-masked networks are built.","marker":"[19]"},{"why":"Provides the zero-extension geometry-transfer baseline for vanilla DeepONet that the paper shows failing on abrupt features.","marker":"[15]"},{"why":"Supplies the masked self-attention operation used to drop points outside the current domain.","marker":"[22]"},{"why":"Supports the result that DeepONet-based preconditioners stay robust across mesh resolutions and iterative schemes.","marker":"[17]"}],"fun_headline_variants":["Masked attention lets DeepONet precondition unseen geometries","Trained on squares, masked-attention DeepONet solves Helmholtz on cracks","Attention masking beats zero-extension for DeepONet preconditioning","Masked self-attention gives DeepONet geometry transfer for Helmholtz","Hybrid solver with masked attention: DeepONet preconditioner transfers to new shapes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that every geometry, no matter how complicated, can be represented adequately by the same fixed 15-by-15 grid, and that interpolating the network input transfers the trained network to the much finer test grids.","fun_headline_variants_meta":{"raw":{"variants":["Masked attention lets DeepONet precondition unseen geometries","Trained on squares, masked-attention DeepONet solves Helmholtz on cracks","Attention masking beats zero-extension for DeepONet preconditioning","Masked self-attention gives DeepONet geometry transfer for Helmholtz","Hybrid solver with masked attention: DeepONet preconditioner transfers to new shapes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000591,"raw_usage":{"total_tokens":2684,"prompt_tokens":771,"completion_tokens":1913,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":387,"completion_tokens_details":{"reasoning_tokens":1819}},"tokens_in":387,"tokens_out":1913,"duration_ms":14079,"temperature":1.0,"reasoning_tokens":1819,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:32:19.473598+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run HINTS-Gauss-Seidel with $N_{\\mathrm{masked}}$ on a domain containing a crack whose width is less than one training-grid spacing ($1/15$) while the test resolution is $h=1/57$; if the crack falls entirely between the 225 fixed training points, the masking matrix $M^\\Omega$ carries no inside-point information there. The experiment would falsify the paper's claim if the method then behaves like the zero-extension $N_{\\mathrm{non-masked}}$ and fails to converge, since it would show the coarse grid, not the mask, is responsible for geometry transfer.","supporting_citations":[{"cited_title":"Zhang, A","cited_arxiv_id":null,"evidence_quote":"Introduces the HINTS hybrid scheme and the vanilla DeepONet preconditioner baseline that this paper extends."},{"cited_title":"and Karniadakis, G.E., 2021","cited_arxiv_id":null,"evidence_quote":"Defines the branch-and-trunk DeepONet structure on which the masked and non-masked networks are built."},{"cited_title":"a nd Pathak, J.,","cited_arxiv_id":null,"evidence_quote":"Provides the zero-extension geometry-transfer baseline for vanilla DeepONet that the paper shows failing on abrupt features."},{"cited_title":"and Polosukhin, I., 2017","cited_arxiv_id":null,"evidence_quote":"Supplies the masked self-attention operation used to drop points outside the current domain."}],"review_version":1}