Pith. sign in

REVIEW 4 major objections 6 minor 35 references

APT, a fused local-graph/global-attention transformer, is claimed to be the first neural operator that learns directly from adaptive-mesh simulations, beating grid- and graph-based operators on five subsurface benchmarks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 01:08 UTC pith:6G246MKG

load-bearing objection A serious empirical architecture paper with honest write-up and genuinely new DMO-native learning, but the marquee ATES comparison is confounded by a training-protocol mismatch. the 4 major comments →

arxiv 2602.11208 v2 pith:6G246MKG submitted 2026-02-10 cs.LG

Adaptive Physics Transformer with Fused Global-Local Attention for Subsurface Energy Systems

Xin Ju , Nok Hei (Hadrian) Fung , Yuyan Zhang , Carl Jacquemyn , Matthew Jackson , Randolph Settgast , Sally M. Benson , Gege Wen This is my paper
classification cs.LG
keywords neural operatoradaptive mesh refinementdynamic mesh optimizationgraph neural operatortransformergated fusionsubsurface flowcross-dataset learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

APT is a neural operator that maps an input permeability/porosity field and a query time directly to pressure, saturation, or temperature fields on any mesh, without moving data to a regular grid. The paper's central claim is that a fused encoder—combining a local graph branch for sharp heterogeneity with a global attention branch for long-range pressure—creates a latent token space that lets the model learn straight from adaptive mesh refinement (DMO/LGR) simulations, avoiding interpolation error. On reported benchmarks, this yields the lowest errors among tested baselines: 0.11% pressure error on faulted CO2 storage (vs 0.20% for MGN-LSTM), 99.0% R2 on an adaptive-mesh aquifer thermal storage task (vs 90.6% for FNO), and competitive basin-scale accuracy with 97% fewer parameters than a nested-FNO cascade. The paper also claims that the mesh-agnostic input lets one model train jointly on datasets with incompatible meshes, lifting a data-limited channelized-reservoir task from 32% to 86.7% R2.

Core claim

The paper's core discovery is that coupling a local Graph Neural Operator (radius-graph pooling with mean aggregation) with a global Perceiver branch (cross-attention onto learnable supernode queries), and merging the two via a learned per-location gate G∈[0,1]^dh to form the latent tokens, is what makes the transformer mesh- and geometry-agnostic. Because the encoder consumes point clouds rather than grid connectivity, APT is the first architecture claimed to train directly on dynamic-mesh-optimization (DMO) simulations and on nested local-grid-refinement (LGR) hierarchies as a single point cloud, without interpolation or per-level submodels. Ablations show the gated fusion matters: global-

What carries the argument

The fused encoder is the load-bearing mechanism. A Global Perceiver Branch projects input nodes onto Ns learnable supernode queries via cross-attention, capturing long-range dependencies without O(N^2) cost; a Local GNO Branch aggregates each node's neighborhood via a fixed-radius graph (radius 20.0 in scaled coordinates, max 128 neighbors, mean aggregation) to honor the sparsity of local interactions. The outputs are combined by Gated Fusion, v_fused = G ⊙ v_attn + (1−G) ⊙ v_gno with a learned gate G∈[0,1]^dh, then projected to N_lat latent tokens processed by DiT blocks under time/conditioning modulation; the decoder cross-attends from arbitrary query points to these tokens. This design ma

Load-bearing premise

The load-bearing premise is that a fixed-radius geometric graph built from node coordinates (radius 20.0 in scaled units, cap 128 neighbors, mean aggregation), together with fixed-size random cell subsampling, captures the heterogeneity that adaptive mesh refinement encodes; if the true mesh topology or refinement structure carries information this graph discards, the claim of learning directly from DMO meshes without interpolation, and the fused branch's advantage, would wea

What would settle it

Run the local branch (and the full APT) on a DMO dataset with the actual mesh connectivity and refinement-level features provided as additional inputs, and also with a fixed gate G=0.5. If either variant beats the coordinate-only radius-graph model or matches the learned gate, the radius-graph and gating claims are falsified. Alternatively, coarsen the DMO meshes while keeping coordinates identical: if prediction error rises steeply, the model is implicitly relying on node density (i.e., on the mesh refinement pattern) rather than on a mesh-agnostic continuous operator.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Direct learning from adaptive meshes removes the interpolation step that degrades sharp fronts; APT reports a relative L2 of 0.011 (99.0% R2) on the ATES adaptive-mesh task, four times lower than grid-interpolated baselines.
  • A single APT model replaces the five-model Nested-FNO cascade on basin-scale CO2 storage with local grid refinement, cutting parameters from 682M to 17.9M at comparable pressure accuracy, and runs full-field inference 6,800× faster than the simulator.
  • On the faulted CO2 storage benchmark, fused APT reaches 0.11% pressure error vs 0.20% for MGN-LSTM, and the ablation shows that neither global-only (0.48%) nor local-only (0.18%) branches match the fused version.
  • Mesh-agnostic inputs allow cross-dataset joint training: training on a Gaussian LGR dataset plus a channelized dataset raises channelized R2 from 32.0% to 86.7%, demonstrating that previously siloed simulation datasets can be combined.
  • Super-resolution behavior differs by data: on smooth car-aerodynamics benchmarks APT improves at full resolution (+0.5%) while baselines degrade by up to 10.2%, whereas on the heterogeneous ATES task APT degrades 1.8%, which the authors attribute to the training data's mesh-not-converged fidelity rather than the architecture.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A testable extension the paper leaves implicit is to visualize the learned gate G across the domain: if G tracks physical scales (favoring local near faults and fronts, global in the far field), that would confirm the intended inductive bias; a fixed 50/50 fusion baseline would isolate how much the learned gating itself contributes.
  • Since the local branch uses a coordinate-radius graph rather than true mesh connectivity, an extension would be to feed refinement level, cell aspect ratio, or cell volume as additional inputs, or to replace the radius graph with the actual mesh graph; if accuracy improves, the paper's DMO-native claim would be strengthened, and if not, the radius graph suffices.
  • The cross-dataset gain suggests APT learns a shared representation of flow physics across geologies; a natural next test is to pre-train on the larger Gaussian dataset and fine-tune on the channelized set, and to probe a third, unseen geomodel class, to see whether the representation transfers as a foundation model.
  • The paper's own caveat—that super-resolution on subsurface data is bounded by the mesh convergence of the training simulations—implies a practical recipe: generate DMO training data at multiple fidelity tolerances and test whether APT's full-resolution accuracy tracks simulator fidelity, which would quantify how much a mesh-agnostic architecture can recover beyond training resolution.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes the Adaptive Physics Transformer (APT), a mesh- and physics-agnostic neural operator that maps an initial condition and a query time to solution fields at arbitrary spatial points. The architecture combines a global Perceiver-style cross-attention encoder with a local graph neural operator (GNO) encoder, fuses the two branches through a learned gate, applies DiT-style latent dynamics, and decodes via cross-attention at query locations. The model is evaluated on seven datasets spanning regular grids, unstructured faulted meshes, dynamic-mesh-optimization (DMO) adaptive meshes, nested local-grid-refinement (LGR) meshes, and a car-aerodynamics benchmark. The central claims are that APT is the first architecture to learn directly from DMO simulations without interpolation, that it outperforms state-of-the-art baselines on subsurface benchmarks, that it shows robust super-resolution behavior on smooth geometries, and that it can pre-train across datasets with incompatible mesh topologies.

Significance. If the claims are established, the paper would be a meaningful step forward for mesh-agnostic operator learning in subsurface applications. The empirical study is broad, internally consistent, and evaluated against external simulators (GEOS, ECLIPSE, IC-FERST) on held-out test sets. The paper is also refreshingly candid about the limitations of super-resolution for subsurface data. However, three load-bearing comparisons currently conflate architectural choices with unrelated factors: the ATES comparison mixes direct one-step prediction with autoregressive rollout, the cross-dataset experiment confounds dataset diversity with training-set size, and the CarBench super-resolution comparison relies on a self-reproduced protocol with a single representative test sample. These issues are addressable and do not invalidate the architecture, but they must be tightened before the headline claims can be accepted.

major comments (4)
  1. [Section 3.3, Appendix C.2.1] The ATES comparison confounds mesh representation with temporal inference strategy. APT is trained in a 'direct one-step' fashion (initial condition plus query time t), while the U-Net and FNO baselines are trained as next-step predictors and rolled out autoregressively on the fixed grid (Appendix C.2.1). Autoregressive rollout accumulates error over 240 steps, so the reported 4x relative-L2 gap in Table 3 cannot be attributed to APT's direct handling of DMO meshes. Notably, for the faulted CO2 benchmark the authors trained U-Net/U-FNO as direct predictors 'to ensure fairness' (Appendix C.1.2); no equivalent control is reported for ATES. Please add a direct-step fixed-grid baseline (e.g., U-Net/FNO with time conditioning) or an autoregressive APT variant, so that the comparison isolates the effect of DMO-native point-cloud learning.
  2. [Section 4.3, Table 6] The cross-dataset training result confounds dataset diversity with total training-data size. 'Channelized only (400)' is compared with 'Gaussian + Channelized (1,400)', so the improvement from 32.0% to 86.7% R2 could simply reflect a 3.5x increase in the number of training samples, not transfer from a different geological distribution. A control trained on 1,400 Channelized-only samples (if the dataset can be augmented or if the original 400 are supplemented with additional channelized simulations) is needed, or at minimum per-dataset learning curves as a function of training-set size. Without such a control, the claim that APT enables 'cross-dataset learning' as opposed to 'more data helps' is not established.
  3. [Section 4.1, Appendix C.1.4, Table 5] The CarBench super-resolution comparison is not a controlled benchmark. APT results were produced from a self-reproduced protocol using the authors' own sampling seed, with three missing cases and no official codebase (as acknowledged in Appendix C.1.4), while all baseline numbers are taken from the original CarBench paper. Table 5 reports only 'a representative unseen test sample' (E S WW WM 648) for CarBench, not the full test set, and the full-test-set APT numbers in Table 13 have large uncertainty (R2 = 96.00 +/- 3.1, Rel L2 = 0.1535 +/- 0.038). The claim that APT is 'the only model to improve at full resolution' is therefore not statistically supported. Please report full-test-set results obtained with identical sampling and evaluation code, or release the code and data so that the comparison can be reproduced.
  4. [Section 2.3, Appendices C.1.2, C.1.3] The claim that APT 'learns directly from DMO' should be more carefully qualified. The local branch uses a fixed-radius geometric graph (radius 20.0, max 128 neighbors, mean aggregation) constructed from cell-center coordinates, not the actual adaptive-mesh connectivity or refinement-level information, and training subsamples a fixed number of cells per step. Thus APT does use raw DMO point clouds without interpolation, which is a defensible reading of 'directly', but the paper does not demonstrate that the DMO resolution information is actually exploited. An ablation comparing the radius graph against the true mesh adjacency graph, or adding refinement level / cell volume as input features, would clarify whether the DMO-specific advantages claimed in Section 3.3 are due to the point-cloud representation itself or to the local graph construction.
minor comments (6)
  1. [Abstract / Section 3 / Appendix C] The paper says 'five subsurface energy benchmarks' in Section 3 but 'seven benchmark datasets' in Appendix C. Please make the count consistent.
  2. [Equation (1)] The gating parameter G is described as 'at each spatial location' but is defined as a single vector in [0,1]^dh. Clarify whether G is shared globally across all locations or is predicted per token / per location.
  3. [Section 3.3, Appendix C.2.1] The fixed-grid interpolation protocol for U-Net/FNO baselines is not fully specified: what is the resolution of the regular voxel grid used for interpolation? Please report it.
  4. [Table 14] The OOD generalization table reports no uncertainties. Given the relatively small gap between APT (R2=0.9107) and FNO (R2=0.9052), standard errors or confidence intervals should be provided.
  5. [Software and Data] The statement that 'the APT model architecture and created datasets will be released upon publication' is welcome, but the CarBench protocol mismatch and the need for reproducibility make it important that code and data be made available to reviewers. Please provide an anonymous repository or detailed enough instructions.
  6. [Throughout] Minor typos include 'Channalized' (Section 4.3), 'dataest' (Appendix C.1.2), and 'For a comprehensive overview' (Appendix C.1.3).

Circularity Check

0 steps flagged

No circularity: results are held-out evaluations; minor same-group baseline citations are not load-bearing.

full rationale

APT is presented as a compositional architecture (Secs. 2.3-2.5) with no fitted parameter that is later renamed as a prediction. All central accuracy claims are computed on held-out test folds against external simulators (GEOS, ECLIPSE, IC-FERST), so the headline results are not forced by construction. The ATES comparison has a training-protocol confound (APT uses direct one-step prediction while U-Net/FNO baselines are rolled out autoregressively, App. C.2.1), but that is a benchmarking fairness issue, not circularity: APT's output is not constructed from baseline errors or from the claimed advantage. Same-group baselines (Nested FNO from Wen et al. 2023a,b; MGN-LSTM from Ju et al. 2024) are used only as comparison checkpoints; these are published methods with public datasets and author-provided checkpoints, and they do not justify APT's predictions. The paper also openly flags its own limitations (e.g., CarBench protocol reproduction, ATES super-resolution data-fidelity bound), which further supports that the authors are not hiding a definitional loop. No equation in the paper reduces to its own input by construction, and the 'first to learn from DMO' claim is a priority statement rather than a derived result. Score 2 reflects only the presence of minor self-citations in baseline selection; there is no substantive circularity.

Axiom & Free-Parameter Ledger

6 free parameters · 5 axioms · 1 invented entities

The paper contributes an architecture, not a derivation; it leans on standard operator-learning approximation theory, simulator-generated ground truth, and several hand-set representation choices (radius graphs, subsampling, supernode counts). No new physical forces, particles, or media are postulated.

free parameters (6)
  • Gating parameter G = learned, in [0,1]^dh
    Eq. (1): v_fused = G * v_attn + (1-G) * v_gno; the fusion weight is the paper's central mechanism and is fitted per-location during training.
  • Radius-graph radius = 20.0 (scaled units)
    Chosen by hand (C.1.2: 'a quarter of the scaled simulation domain'; C.1.3: 'one-tenth'); controls the local receptive field and is load-bearing for the local branch.
  • Max neighbors per graph pooling = 128
    Capped in C.1.3 ('maximum number of neighboring cells in each graph pooling operation is capped at 128'); hand-set.
  • Supernode counts = 256 (2D), 1024 (CarBench), 8192 (3D), 4096 (pooling)
    Selected via 'a small grid search on the validation set' (C.1.3) or heuristically; controls global-branch capacity.
  • Training subsample size per case = 1,024 to 262,144 cells depending on dataset
    Fixed-size random subsampling chosen for memory stability (C.1.2, C.1.3); the model never sees the full mesh at once.
  • Latent tokens and hidden dims = 128-1024 tokens; dh 48-192
    Per-dataset architecture choices tuned on validation; these set the representational ceiling of the latent dynamics.
axioms (5)
  • standard math Operator universal approximation: neural operators can approximate the solution operator G†
    Sections 2.1-2.2 frame the task as learning G_theta ~ G†, relying on operator-learning theory (Kovachki, Lanthaler, Li et al. 2020).
  • domain assumption Radius-graph mean aggregation captures local differential-operator behavior
    Sec. 2.3: 'the local branch (v_gno) adheres to the sparsity inherent in differential operators'; edges are built from coordinates, not the true mesh connectivity.
  • domain assumption Per-sample z-score normalization preserves learnable physical structure
    Eq. (17) normalizes each field per case; errors are thus computed on normalized fields, and absolute pressure magnitude is removed from the loss.
  • domain assumption Simulator outputs (GEOS, ECLIPSE, IC-FERST) are adequate ground truth
    All training and test labels come from these simulators (C.1.1-C.2.3); any simulator error transfers to the surrogate.
  • domain assumption Fixed-size random cell subsampling represents each mesh state sufficiently
    Training samples only 1,024 to 262,144 of up to ~1M cells per case (C.1.2-C.1.3); the decoder is trained on the same subsample distribution.
invented entities (1)
  • Anchor supernodes (ATES implementation) no independent evidence
    purpose: Deterministic point-selection mechanism to preserve critical boundary and well regions when input node counts vary with DMO
    An architecture component, not a new physical entity; it does not carry an independently falsifiable handle.

pith-pipeline@v1.3.0-alltime-deepseek · 27015 in / 15088 out tokens · 143093 ms · 2026-08-03T01:08:45.253575+00:00 · methodology

0 comments
read the original abstract

The Earth's subsurface is a cornerstone of modern society, providing essential energy resources like hydrocarbons, geothermal, and minerals while serving as the primary reservoir for $CO_2$ sequestration. However, full physics numerical simulations of these systems are notoriously computationally expensive due to geological heterogeneity, high resolution requirements, and the tight coupling of physical processes with distinct propagation time scales. Here we propose the $\textbf{Adaptive Physics Transformer}$ (APT), a geometry-, mesh-, and physics-agnostic neural operator that explicitly addresses these challenges. APT fuses a graph-based encoder to extract high-resolution local heterogeneous features with a global attention mechanism to resolve long-range physical impacts. Our results demonstrate that APT outperforms state-of-the-art architectures in subsurface tasks across both regular and irregular grids with robust super-resolution capabilities. Notably, APT is the first architecture that learns directly from HR-adaptive mesh refinement simulations. We also demonstrate APT's favorable scaling behavior and cross-dataset learning capability, positioning it as a robust and scalable backbone for large-scale subsurface foundation model development.

Figures

Figures reproduced from arXiv: 2602.11208 by Carl Jacquemyn, Gege Wen, Matthew Jackson, Nok Hei (Hadrian) Fung, Randolph Settgast, Sally M. Benson, Xin Ju, Yuyan Zhang.

Figure 1
Figure 1. Figure 1: Architectural Overview. The fused encoder combines a (a) Global Perceiver Encoder that projects input features onto supernode queries via cross-attention with a (b) Local GNO Encoder that aggregates neighborhood information through radius graph pooling. A (c) Gated Fusion Mechanism adaptively combines global (vattn) and local (vgno) representations via a learned gate G ∈ [0, 1]d , followed by DiT blocks an… view at source ↗
Figure 2
Figure 2. Figure 2: Benchmark datasets for evaluating APT across diverse subsurface applications. (a) to (d) each displays input parameter fields (e.g., permeability) alongside the temporal evolution of output fields (e.g., saturation, pressure, temperature). (a) 2D Geologic Carbon Storage (GCS) utilizing an irregular mesh to resolve complex fault geometries. (b) 2D Hydrocarbon extraction on a Cartesian mesh with varying well… view at source ↗
Figure 3
Figure 3. Figure 3: Visualization of prediction error and state evolution across different subsurface applications. Rows compare APT against baseline models (MGN-LSTM, U-FNO, and FNO) at multiple time steps (t). (a) Saturation error maps (δSg) for the 2D irregular mesh geologic carbon storage dataset, showing APT’s stability near complex fault geometries. (b) Water saturation error maps (δSw) for the 2D Cartesian hydrocarbon … view at source ↗
Figure 4
Figure 4. Figure 4: Cross-dataset training setup. (a) Gaussian permeability fields defined on nested grids with 4 levels of LGRs. (b) Channel￾ized permeability fields LGR defined with 3 levels of LGRs. Experiment Set Up. We combine two CO2 storage datasets with different semi-adaptive resolutions: (1) a sub￾set of the open source dataset (Wen et al., 2023b) with 4 levels of nested grids, and (2) our created Channelized datase… view at source ↗
Figure 5
Figure 5. Figure 5: Heterogeneous permeability realizations with two fixed impermeable faults and one injection well for three cases. The well coordinates for each case are shown at the top, with insets displaying an enlarged view of the well vicinity. APT Implementation and Training Details During each training step, input grids are sampled with a fixed size of 1024 cells. The APT model has a total of 1.38M parameters. The a… view at source ↗
Figure 6
Figure 6. Figure 6: Comparison of model generalizability for gas saturation predictions. Predictions at 950 days are shown across five distinct test meshes with varied permeability fields and well locations. The rows, from top to bottom, display: (1) reservoir permeability, (2) the high-fidelity (HF) simulation ground truth, (3) the APT prediction, (4) the APT prediction error, and (5) the MGN-LSTM prediction error [PITH_FUL… view at source ↗
Figure 7
Figure 7. Figure 7: Comparison of model generalizability for pore pressure predictions. Predictions at 950 days are shown across five distinct test meshes with varied permeability fields and well locations. The rows, from top to bottom, display: (1) reservoir permeability, (2) the high-fidelity (HF) simulation ground truth, (3) the APT prediction, (4) the APT prediction error, and (5) the MGN-LSTM prediction error. C.1.3. 3D … view at source ↗
Figure 8
Figure 8. Figure 8: APT framework for multiresolution CO2 storage modeling. (a) Input fields include heterogeneous permeability, spatial coordinates, injection conditions (e.g., Well 4), and cell volumes across LGR levels. (b) A single, unified APT model processes all multiscale fields and scalar parameters end-to-end. (c) The model outputs predictions, such as the gas saturation field at year 30, visualized across all refine… view at source ↗
Figure 9
Figure 9. Figure 9: APT model’s predictions for gas saturation fields at year 30 from a 3-well setting. Each row corresponds to a different LGR resolution level, and each column shows a different test case. From top to bottom: (1) Permeability fields and LGR meshes, (2) APT-predicted gas saturation (Sg), (3) Ground truth saturation from ECLIPSE simulations, and (4) prediction errors [PITH_FULL_IMAGE:figures/full_fig_p023_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Temporal evolution of the gas saturation plume for Well 2. The first and second rows respectively show the gas saturation (Sg) fields predicted by APT and the ECLIPSE simulations (HF) for Well 2 at five different times. The third row shows the saturation error (δsg) between APT and HF. 23 [PITH_FULL_IMAGE:figures/full_fig_p023_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Qualitative comparison of surface pressure predictions for design E S WW WM 648 from the unseen test set of the DrivAer￾Net++ dataset (isometric view). Each row compares predicted pressure fields (top) and absolute error maps (bottom) against the CFD ground truth. APT produces visually smooth predictions across the surface. Note that the baseline results are reproduced from (Elrefaie et al., 2025), wherea… view at source ↗
Figure 12
Figure 12. Figure 12: Qualitative comparison of surface pressure predictions for design E S WW WM 648 from the unseen test set of the DrivAer￾Net++ dataset (side view). Each row compares predicted pressure fields (top) and absolute error maps (bottom) against the CFD ground truth. APT produces visually smooth predictions across the surface. Note that the baseline results are reproduced from (Elrefaie et al., 2025), whereas APT… view at source ↗
Figure 13
Figure 13. Figure 13: OOD evaluation setup. The model is trained on a mixture of continuous/discontinuous Gaussian and continuous von Karman fields, but tested on a held-out class of discontinuous von Karman fields, requiring generalization to unseen geological structures [PITH_FULL_IMAGE:figures/full_fig_p028_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Permeability distribution of the channelized reservoir model. The histogram reveals a distinct bimodal signature: a low￾permeability background facies (ln(k) ≈ 3.5) and high-permeability channel sands (ln(k) ≈ 6.5). This sharp contrast approximates binary geological media, testing the model’s ability to preserve discontinuous interfaces. APT implementation. To facilitate the multi-dataset training, we use… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

35 extracted references · 14 linked inside Pith

  1. [2]

    13 Adaptive Physics Transformer for Subsurface Energy Systems B

    computational complexity, where N represents the number of input points/nodes, which prevents direct application to large-scale physical systems. 13 Adaptive Physics Transformer for Subsurface Energy Systems B. Architectural Details and Evaluation Metrics B.1. Unified Modulation for Temporal and Conditional Inputs To enable the model to adapt to varying t...

    2023

  2. [7]

    doi: 10.1016/j.neunet

    ISSN 18792782. doi: 10.1016/j.neunet. 2021.11.022. URL https://doi.org/10.1016/ j.neunet.2021.11.022. Hamilton, W., Ying, Z., and Leskovec, J. Inductive repre- sentation learning on large graphs. Advances in Neural Information Processing Systems, 30,

    doi:10.1016/j.neunet 2021

  3. [8]

    APT Training Efficiency Metrics. Dataset Model Size Input Cells Throughput (samples/sec) Effective Batch Memory/M Params (GB) Faulted 1.38 M 1,024 ∼371 32 2.33 Hydrocarbon 12 M 32,768 ∼22 32 3.65 Car Bench 6.70 M 10,000 ∼36 64 2.51 ATES 17 M 8,192 ∼12 128 0.33 Basin-scale GCS 17.9 M 262,144 ∼2.2 32 1.33 C. Benchmark Datasets We evaluate APT on seven bench...

    2025

  4. [9]

    URL http: //arxiv.org/abs/2304.13221. Lee, J. E., Zhu, M., Xi, Z., Wang, K., Yuan, Y . O., and Lu, L. Efficient and generalizable nested fourier-deeponet for three-dimensional geological carbon sequestration. Engi- neering Applications of Computational Fluid Mechanics, 18(1):2435457,

    Pith/arXiv arXiv

  5. [10]

    Fourier neural operator for parametric partial differential equa- tions

    Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhat- tacharya, K., Stuart, A., and Anandkumar, A. Fourier neural operator for parametric partial differential equa- tions. arXiv preprint arXiv:2010.08895, 2020a. Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhat- tacharya, K., Stuart, A., and Anandkumar, A. Multipole graph neural operator for pa...

    Pith/arXiv arXiv 2010

  6. [11]

    Each row compares predicted pressure fields (top) and absolute error maps (bottom) against the CFD ground truth

    Qualitative comparison of surface pressure predictions for design E S WW WM 648 from the unseen test set of the DrivAer- Net++ dataset (isometric view). Each row compares predicted pressure fields (top) and absolute error maps (bottom) against the CFD ground truth. APT produces visually smooth predictions across the surface. Note that the baseline results...

    2025

  7. [12]

    Lu, L., Jin, P., and Karniadakis, G. E. Deeponet: Learning nonlinear operators for identifying differential equations based on the universal approximation theorem of opera- tors. arXiv preprint arXiv:1910.03193,

    Pith/arXiv arXiv 1910

  8. [13]

    Pfaff, T., Fortunato, M., Sanchez-Gonzalez, A., and Battaglia, P. W. Learning mesh-based simulation with graph networks. arXiv preprint arXiv:2010.03409,

    Pith/arXiv arXiv 2010

  9. [14]

    U-net: Con- volutional networks for biomedical image segmentation

    Ronneberger, O., Fischer, P., and Brox, T. U-net: Con- volutional networks for biomedical image segmentation. In Medical Image Computing and Computer-Assisted Intervention–MICCAI 2015: 18th International Confer- ence, Munich, Germany, October 5-9, 2015, Proceedings, Part III 18, pp. 234–241. Springer,

    2015

  10. [15]

    org/abs/2403.07536

    URL http://arxiv. org/abs/2403.07536. Sun, Z., Yang, Y ., and Yoo, S. A neural pde solver with temporal stencil modeling. In International Conference on Machine Learning, pp. 33135–33155. PMLR,

    Pith/arXiv arXiv

  11. [16]

    Unisoma: A unified transformer-based solver for multi-solid systems

    11 Adaptive Physics Transformer for Subsurface Energy Systems Tao, S., Feng, Z., Sun, H., Zhu, Z., and Liu, Y . Unisoma: A unified transformer-based solver for multi-solid systems. arXiv preprint arXiv:2506.06021,

    Pith/arXiv arXiv

  12. [17]

    N., Kaiser,Ł., and Polosukhin, I

    Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser,Ł., and Polosukhin, I. Attention is all you need. Advances in Neural Information Process- ing Systems, 2017-Decem(Nips):5999–6009,

    2017

  13. [18]

    Wen, G., Tang, M., and Benson, S

    URL http://arxiv.org/abs/2405.13998. Wen, G., Tang, M., and Benson, S. M. Towards a predic- tor for co2 plume migration using deep neural networks. International Journal of Greenhouse Gas Control, 105: 103223,

    Pith/arXiv arXiv

  14. [19]

    Wen, G., Li, Z., Long, Q., Azizzadenesheli, K., Anandku- mar, A., and Benson, S. M. Real-time high-resolution co 2 geological storage prediction using nested fourier neural operators. Energy & Environmental Science, 16 (4):1732–1741, 2023a. Wen, G., Li, Z., Long, Q., Azizzadenesheli, K., Anandku- mar, A., and Benson, S. M. Real-time high-resolution co 2 g...

    Pith/arXiv arXiv

  15. [20]

    Tran- solver++: An accurate neural solver for pdes on million- scale geometries

    Wu, H., Luo, H., Wang, H., Wang, J., and Long, M. Tran- solver++: An accurate neural solver for pdes on million- scale geometries. arXiv preprint arXiv:2502.02414,

    Pith/arXiv arXiv

  16. [25]

    FNO and U-FNO results are from Badawi & Gildin (2025)

    Baselines. FNO and U-FNO results are from Badawi & Gildin (2025). Both baselines use width 64, 10 Fourier modes, and are trained with relative L2 loss including gradient terms. C.1.2. I RREGULAR GRID CO 2 STORAGE Governing Equations. We consider a multi-phase flow problem with CO2 and water in the context of geological storage of CO2. The system’s dynamic...

    2025

  17. [27]

    (previously SOTA on this dataest) to operate directly on the unstructured mesh without interpolation. MGN-LSTM is trained autoregressively on 19-step sequences, while MGN is trained for single- step prediction; both generate the full 950-day sequence via autoregressive rollout at inference. We use the checkpoints and noise injection strategy from Ju et al...

    2024

  18. [28]

    Since the PEBI mesh is unstructured with varying topology across cases, the simulation data must be interpolated onto a regular grid before training

    and U-FNO (Wen et al., 2022), however, require structured grid inputs. Since the PEBI mesh is unstructured with varying topology across cases, the simulation data must be interpolated onto a regular grid before training. As the average cell number of this dataset is around 1600, we use a resolution of 40 × 40 to interpolate the field using linear interpol...

    2022

  19. [29]

    All models are evaluated with batch size 1 on an NVIDIA A100-SXM4-80GB GPU under identical inference settings

    Quantitative comparison on the CarBench’s full test set across various neural surrogate models. All models are evaluated with batch size 1 on an NVIDIA A100-SXM4-80GB GPU under identical inference settings. Following the measurement recommended in (Elrefaie et al., 2025), uncertainties are rounded to two significant digits and central values are rounded t...

    2025

  20. [30]

    32.58 1.50 8.29 90.25 ± 0.22 0.2436 ± 0.0013 RegDGCNN(Elrefaie et al., 2024a) 1.44 27.11 231.98 93.27 ± 0.33 0.2006 ± 0.0016 PointTransformer(Zhao et al.,

    2006

  21. [34]

    Each row compares predicted pressure fields (top) and absolute error maps (bottom) against the CFD ground truth

    Qualitative comparison of surface pressure predictions for design E S WW WM 648 from the unseen test set of the DrivAer- Net++ dataset (side view). Each row compares predicted pressure fields (top) and absolute error maps (bottom) against the CFD ground truth. APT produces visually smooth predictions across the surface. Note that the baseline results are ...

    2025

  22. [35]

    A vertical injection well with a radius of 0.1 m delivers wastewater at a constant rate into a radially symmetric system x(r, z)

    to develop the wastewater injection dataset. A vertical injection well with a radius of 0.1 m delivers wastewater at a constant rate into a radially symmetric system x(r, z). The well completion may span the full reservoir thickness or be restricted to a selected depth interval. Here, the reservoir thickness ranges from 125 to 500 m, with no-flow boundary...

    2022

  23. [192]

    We found that using only 1024 supernodes for this dataset suffices to achieve good performance

    The network depth is configured with 2 transformer blocks in the encoder, 2 blocks in the approximator, and 4 blocks in the decoder (each containing paired self-attention and cross-attention mechanisms). We found that using only 1024 supernodes for this dataset suffices to achieve good performance. The input projection layer maps din = 3 features (corresp...

    2017

  24. [2010]

    B., and Stuart, A

    Bhattacharya, K., Hosseini, B., Kovachki, N. B., and Stuart, A. M. Model reduction and neural networks for paramet- ric pdes. arXiv preprint arXiv:2005.03180,

    Pith/arXiv arXiv 2005

  25. [2014]

    Drivaernet: A parametric car dataset for data-driven aerodynamic de- sign and graph-based drag prediction

    Elrefaie, M., Dai, A., and Ahmed, F. Drivaernet: A parametric car dataset for data-driven aerodynamic de- sign and graph-based drag prediction. arXiv preprint arXiv:2403.08055, 2024a. Elrefaie, M., Morar, F., Dai, A., and Ahmed, F. Drivaer- net++: A large-scale multimodal car dataset with com- putational fluid dynamics simulations and deep learning benchm...

    Pith/arXiv arXiv

  26. [2015]

    Intuitively, CNNs were used for regular Cartesian discretizations (Zhu & Zabaras, 2018; Mo et al., 2019; Wen et al., 2022; Tang et al., 2022; Wen et al., 2023a; 2021)

    were among the first types of architectures adapted for flow prediction tasks in the subsurface. Intuitively, CNNs were used for regular Cartesian discretizations (Zhu & Zabaras, 2018; Mo et al., 2019; Wen et al., 2022; Tang et al., 2022; Wen et al., 2023a; 2021). However, CNN’s inherent reliance on fixed computational stencils and structured grid topolog...

    2018

  27. [2016]

    Since PEBI meshes are orthogonal by construction, the flow simulations can be performed with a TPFA-based finite-volume scheme without compromising solution accuracy

    generated with the MATLAB Reservoir Simulation Toolbox (MRST) to mesh a 1 km × 1 km × 1 m domain containing an injector well and two straight impermeable faults that are conformal with the grid. Since PEBI meshes are orthogonal by construction, the flow simulations can be performed with a TPFA-based finite-volume scheme without compromising solution accur...

    2024

  28. [2017]

    Predicting physics in mesh-reduced space with temporal attention

    Han, X., Gao, H., Pffaf, T., Wang, J.-X., and Liu, L.-P. Predicting physics in mesh-reduced space with temporal attention. arXiv preprint arXiv:2201.09113,

    Pith/arXiv arXiv

  29. [2018]

    Related Work A.1

    12 Adaptive Physics Transformer for Subsurface Energy Systems A. Related Work A.1. Convolutional Neural Network (CNN) CNNs, especially U-Net (Krizhevsky et al., 2012; Ronneberger et al.,

    2012

  30. [2020]

    Message pass- ing neural pde solvers

    Brandstetter, J., Worrall, D., and Welling, M. Message pass- ing neural pde solvers. arXiv preprint arXiv:2202.03376,

    Pith/arXiv arXiv

  31. [2021]

    3.05 6.65 95.68 93.59 ± 0.54 0.1909 ± 0.0024 TripNet(Chen et al.,

    1909

  32. [2022]

    B., Levine, M

    Calvello, E., Kovachki, N. B., Levine, M. E., and Stu- art, A. M. Continuum Attention for Neural Operators. jun 2024a. URL http://arxiv.org/abs/2406. 06486. Calvello, E., Kovachki, N. B., Levine, M. E., and Stuart, A. M. Continuum attention for neural operators. arXiv preprint arXiv:2406.06486, 2024b. Chen, Q., Elrefaie, M., Dai, A., and Ahmed, F. Tripnet...

    arXiv

  33. [2023]

    Bartolucci, F., Raonic, B., Molinaro, R., de B ´ezenac, E., Mishra, S., and Alaifari, R

    ISSN 10495258. Bartolucci, F., Raonic, B., Molinaro, R., de B ´ezenac, E., Mishra, S., and Alaifari, R. The false promise of zero- shot super-resolution in machine-learned operators. arXiv preprint arXiv:2510.06646,

    arXiv

  34. [2024]

    Ab-upt: Scaling neural cfd surrogates for high-fidelity automotive aero- dynamics simulations via anchored-branched universal physics transformers

    Alkin, B., Bleeker, M., Kurle, R., Kronlachner, T., Sonnleit- ner, R., Dorfer, M., and Brandstetter, J. Ab-upt: Scaling neural cfd surrogates for high-fidelity automotive aero- dynamics simulations via anchored-branched universal physics transformers. arXiv preprint arXiv:2502.09692,

    arXiv

  35. [2025]

    Neural operators with localized integral and differential kernels

    Liu-Schiaffini, M., Berner, J., Bonev, B., Kurth, T., Aziz- zadenesheli, K., and Anandkumar, A. Neural operators with localized integral and differential kernels. arXiv preprint arXiv:2402.16845,

    Pith/arXiv arXiv