Pith. sign in

REVIEW 4 major objections 6 minor 14 references

Agents4GEOS turns plain-English requests into validated GEOS CO2-sequestration simulations and trains a GNN surrogate that tracks plume migration.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 15:01 UTC pith:XG7TRRFU

load-bearing objection Well-engineered, honest engineering description whose central validation claim outruns the evidence. the 4 major comments →

arxiv 2607.18557 v1 pith:XG7TRRFU submitted 2026-07-20 physics.geo-ph

AGENTS4GEOS: agentic platform for open-source multi-physics simulation

Adriano M. A. C\^ortes , Roberto M. Velho , Fernando A. Rochinha , Alvaro L. G. A. Coutinho , Mauricio Araya-Polo , Herv\'e Gross This is my paper
classification physics.geo-ph
keywords agentic AIGEOS simulatorCO2 sequestrationgraph neural networkssurrogate modelsreservoir simulationModel Context Protocolphysics-informed ML
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper claims that a multi-agent AI system called Agents4GEOS can lower the barrier to building physically consistent, validated multiphysics simulations of subsurface CO2 storage. The system takes a natural-language request, decomposes it into specialized subagents, and produces a schema-valid GEOS input file through real computations of fluid properties, meshes, and XML assembly — not free-form text. As its main demonstration, the agents reproduce the PUNQ-S3 no-hysteresis CO2-sequestration benchmark from a one-sentence prompt, and the resulting GEOS runs are compared against published ECLIPSE and MRST results. The same pipeline generates 200 permeability-varying cases on which the Plumecast graph neural network is trained, and the surrogate's 100-year CO2 saturation fields match the high-fidelity GEOS ground truth well. If correct, this enables large, validated simulation datasets and surrogate models for uncertainty quantification and many-query reservoir workflows, at a fraction of the usual manual effort.

Core claim

Agents4GEOS is an agentic framework built on the Model Context Protocol in which agents plan, tools compute, and knowledge modules encode domain expertise. The paper's central demonstration is that a user asking, in plain English, to reproduce a published PUNQ-S3 CO2-sequestration study can receive, in a single session, a schema-valid, physics-checked GEOS XML deck that reproduces the published saturation behavior at the three reference observation points — with the system openly flagging an injection-rate discrepancy it could not reconcile between the two source papers. The paper further claims that the 200-case dataset produced this way is validated against published ECLIPSE and MRST resul

What carries the argument

The load-bearing mechanism is the strict separation between the agent layer (eleven slash-command agents plus four fresh-context subagents that return typed JSON contracts), the tool layer (52 stateless MCP tools grouped into six scientific domains and backed by real computation libraries such as pyResToolbox and PyVista), and the knowledge modules (seven Python modules encoding GEOS field names, fluid models, cross-references, sanity rules, unit conventions, formatting, and preprocessing, distilled from an audit of 200+ official GEOS input files). Orchestration patterns — pipeline, fan-out, feedback loop, and quality contract — coordinate these pieces, with a fresh-context independent revie

Load-bearing premise

That representing the PUNQ-S3 reservoir as an immiscible dead-oil fluid (ignoring gas dissolution) and shifting it from roughly 2340 m to 840 m depth does not materially change the CO2 plume and water dynamics that the benchmark is meant to capture.

What would settle it

Run both the agent-generated dead-oil GEOS deck and a full compositional CO2-brine GEOS deck on the same mesh and depth, and compare the saturation fields at the three PUNQ-S3 observation cells over the 500-year horizon; a clear difference in plume shape, crest saturation, or flank draining times would falsify the claim that the approximation preserves the main flow dynamics.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Natural-language-to-GEOS workflows could let domain experts author and audit complex multiphysics simulations without hand-writing hundreds of lines of XML.
  • Agent-built datasets, validated against published benchmarks, provide a controlled source of training data for GNN surrogates of CO2 plume migration.
  • The Plumecast results suggest physics-informed features — transmissibility-aware edges, tabulated relative permeabilities, Kozeny-Carman porosity — improve long-horizon surrogate accuracy and reduce false plume spreading.
  • The reusable learning loop that persists runtime-error lessons into knowledge modules reduces the chance of repeating the same mistakes across future simulation decks.
  • The system's capability-tier model routing indicates that cost-aware agent architectures can keep token budgets manageable while maintaining reliability on complex tasks.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the dead-oil/depth-relocation equivalence generalizes, agent-driven approximations could become a standard, reported practice in benchmark reproduction — but the unvalidated equivalence is a risk that should be tested on other reservoirs before broad claims about fidelity are made.
  • The same separation-of-concerns architecture could transfer to other XML-driven multiphysics simulators beyond GEOS, such as thermal-hydraulic or geomechanics codes, whenever schema-valid input generation and dataset curation are bottlenecks.
  • The system's habit of surfacing the unreconciled injection-rate discrepancy rather than silently picking a number is a model for agentic scientific tools that prioritize transparency over apparent smoothness.
  • A direct stress test: generate the same PUNQ-S3 case with hysteresis and compare the agent's relative-permeability endpoint fit against the raw published curves — if the fit fails to reproduce hysteresis behavior, the fidelity claim narrows to the no-hysteresis setting.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper presents Agents4GEOS, a Model Context Protocol (MCP) based multi-agent system with 52 deterministic tools, eleven slash-command agents and four fresh-context subagents, plus knowledge modules distilled from more than 200 GEOS input files. The stated goal is to lower the barrier to producing schema-valid, physically consistent GEOS simulation decks from natural-language prompts. The central demonstration is a single conversational session that reproduces a PUNQ-S3 no-hysteresis CO2 storage benchmark from a one-sentence request, followed by the generation of 200 permeability-perturbed simulations, training of the Plumecast GNN surrogate, and a visual comparison of predicted and simulated CO2 saturation at 100 years. The paper claims in Section 4 that 'validation against published ECLIPSE and MRST results confirms the fidelity of the GEOS simulations produced by Agents4GEOS.'

Significance. If the validation claim can be made quantitative and the modeling assumptions tested, the contribution is significant: it would reduce a real bottleneck in generating large training datasets for GNN surrogates on unstructured meshes. The architecture has genuine strengths: a strict separation between agents (decisions), tools (computation), and knowledge modules (domain patterns); structured JSON contracts for subagent handoffs; an independent fresh-context reviewer; and grounding of every returned quantity in actual computation. These are valuable design choices worth publishing. However, the evidence that the produced GEOS simulations reproduce the published benchmarks is currently mainly qualitative, and the manuscript itself flags the two most load-bearing limitations: the dead-oil/depth modeling assumption and the endpoint-fitted reversible relative-permeability curve. Those limitations need to be resolved before the central claim can be accepted.

major comments (4)
  1. [Section 4 and Figure 7] The sentence 'Validation against published ECLIPSE and MRST results confirms the fidelity ...' is the central evidence, but the only benchmark comparison is a three-point saturation-versus-time figure described as 'matching perfectly.' No error metrics are reported, no curve is identified as coming from ECLIPSE ([12]) versus MRST ([11]), and no spatial comparison of full 3D saturation fields is shown. Please add quantitative metrics (e.g., RMSE or max error per observation point), overlay the reference curves on Figure 7, and state explicitly which source each curve is from. As written, the reader cannot distinguish a genuine quantitative match from a favorable visual impression.
  2. [Section 3.1] Two modeling changes are introduced before the benchmark comparison: the compositional CO2-brine system is replaced by immiscible dead-oil, and the formation is relocated from approximately 2340 m to 840 m. The text states this 'is assumed not to interfere with the exploration of the main aspects of the flow dynamics,' but the assumption is never tested. The injection-rate discrepancy between the two sources (18 m3/day per well versus 0.15 pore volumes in 10 years) is also unresolved; the agent kept the literal value, but the sensitivity of the results to the alternative value is not explored. Because these choices directly condition the validation claim, the paper should quantify their impact, for example by running the original depth/compositional configuration or a rate-sensitivity study.
  3. [Section 3.1] The 'no-hysteresis signature' in Figure 7 is largely imposed by construction. The agent fits a reversible Brooks-Corey curve using only the published endpoints (Swc = 0.31, Sg,max = 0.69), and a reversible curve cannot exhibit hysteresis. The paper acknowledges that the curve is 'an endpoint fit rather than the raw published curve,' but then Figure 7 tests only that fit, not the physical fidelity of the simulation. Please compare the fitted curve against the full published drainage relative-permeability data over the saturation range, and examine the sensitivity of the saturation histories to the fitted exponent and endpoints.
  4. [Section 3.2 and Figure 11] The Plumecast surrogate comparison is qualitative only: no error metric is reported, and the dataset-generation protocol is underspecified. In particular, the permeability sampling range and distribution, the construction of the 100/100 train/test split, and whether the displayed case belongs to the test set are not stated. Since the paper presents the 200-simulation dataset as a contribution for GNN training, quantitative evaluation over the test set (e.g., RMSE of CO2 saturation fields, plume-area error as a function of time) is needed. Note also that comparing a GEOS-trained surrogate with GEOS validates surrogate accuracy, not the physical fidelity of GEOS itself; the latter must rest on the benchmark comparison.
minor comments (6)
  1. [Section 3.1] Typo: '18 rm3/day' should be '18 m3/day'. Also, Figures 9 and 10 use 'PUNQ-3D' while the text uses 'PUNQ-S3'; please make the nomenclature consistent.
  2. [Section 3.2] The placeholder '(ADD REFS)' appears after 'PyG and PGT'; unresolved references must be completed before publication.
  3. [Figure 7 caption] The caption says 'cf. Figure 6 of [12]', but Section 4 claims validation against both ECLIPSE and MRST results. Specify in the caption which reference curve (ECLIPSE or MRST) is being compared, and whether Figure 7 includes data from both sources.
  4. [Figures 9 and 10] Figures 9 and 10 are not discussed in the main text. Either reference them explicitly in Section 3.2 or remove them to avoid dangling figures.
  5. [General] Given the paper's emphasis on open-source software and reproducibility, please add a code/data availability statement with a repository URL and version/commit identifier for Agents4GEOS, the GEOS version used, and the 200-run dataset generation seeds.
  6. [Section 3.1] Minor typo: in the decision-gate text, 'COz-brine' should be 'CO2-brine'.

Circularity Check

1 steps flagged

The no-hysteresis validation signature is inserted by construction; the agent platform itself is not circular.

specific steps
  1. fitted input called prediction [Section 3.1, Agents4GEOS in Action (input-deck build and post-processing paragraphs; Fig. 7 caption)]
    "a Brooks–Corey fit to the published drainage endpoints (a reversible curve, hence inherently hysteresis-free) ... residual trapping identically zero at all times ... matching perfectly the results of [12]."

    The reversible Brooks–Corey curve is an input fitted to the published endpoints. A reversible relative-permeability model mathematically cannot generate residual trapping or scanning hysteresis, so the output's 'residual trapping identically zero' and the flank-drainage behavior are consequences of the chosen constitutive input, not emergent predictions. Reporting this as 'matching perfectly' the no-hysteresis reference validates the input choice rather than an independent simulation discovery. The plume shape and timing at the observation cells are still real simulation results, so only the hysteresis-free aspect is forced by construction.

full rationale

The core contribution — an MCP-based agentic layer, 52 tools, knowledge modules, and XML assembly/validation — is an engineering system and is not derived from its outputs; none of that is circular. The only constructed 'prediction' is the no-hysteresis signature: the agent's choice of a reversible Brooks–Corey curve directly forces zero residual trapping and reversible drainage, so Figure 7's 'no-hysteresis signature' is partly a tautology. The remaining agreement at the three observation cells is a legitimate but purely qualitative reproduction, with no numerical error metrics. The Plumecast evaluation is standard supervised testing on GEOS-generated data and is not circular, though it is not external validation. The untested dead-oil and depth-relocation assumptions are correctness risks, not circularity. Self-citations [7,8] are architectural credits and are not load-bearing. Overall, the platform itself is not circular; the central validation claim has one component reduced by construction, yielding a moderate score.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

The central demonstration rests on fitted rel-perm endpoints, unspecified porosity-permeability scaling, unstated sample ranges, and the untested assumption that an LLM harness and its knowledge modules produce faithful GEOS decks. These are engineering choices, not fitted constants in a scientific derivation.

free parameters (3)
  • Brooks–Corey relative-permeability exponent and endpoint fit = Swc=0.31, Sg,max=0.69; exponent not stated
    In §3.1, rel-perm curves are a Brooks–Corey fit to published drainage endpoints; only endpoints are given, so the curve shape used in the reproduction is a free choice. The reversible curve ensures the no-hysteresis signature, making the match to [12] partly by construction.
  • Kozeny–Carman porosity-permeability scaling = unstated
    In §3.2/§4, 200 realizations use porosity 'derived through the Kozeny–Carman relationship'; the scaling constant/exponent is not specified, so the dataset is not reproducible.
  • Permeability sampling range and distribution = unstated
    §3.2: 'sampled variations of the permeability spatial field' over 'a representative permeability range'; no distribution, range, or seed is given.
axioms (5)
  • domain assumption GEOS is an accurate multi-physics simulator for this CO2-brine flow regime
    The paper cites [10] for GEOS; all subsequent physical-fidelity claims depend on GEOS's solver correctness.
  • domain assumption ECLIPSE and MRST reference results are valid ground truth for PUNQ-S3
    §3.1/§4 compare GEOS output to [11,12] without independent verification of those references.
  • ad hoc to paper The dead-oil representation and the 840 m depth relocation preserve the benchmark's flow dynamics
    §3.1: the agent recommended and the user accepted modeling the system as immiscible dead-oil and relocating the formation to 840 m; the paper says this 'is assumed not to interfere' with capturing primary storage mechanisms.
  • domain assumption Fresh-context reviewer and sanity rules catch physically significant errors
    §2.4/§3.1: the system relies on geos-reviewer and knowledge modules for quality, yet GEOS rejected the first two decks, showing this assumption is only partially true.
  • domain assumption LLM agents (Claude Code plus MCP) execute tools and JSON contracts faithfully
    The entire pipeline depends on the harness's reliability; no adversarial or robustness testing is presented.

pith-pipeline@v1.3.0-alltime-deepseek · 10301 in / 13769 out tokens · 140684 ms · 2026-08-01T15:01:38.707161+00:00 · methodology

0 comments
read the original abstract

Multi-physics simulations are essential for understanding and monitoring intricate subsurface processes such as CO2 storage. Their computational demands call for surrogate models and, for unstructured meshes, Graph Neural Networks (GNNs) are natural candidates. The main bottleneck in developing them is generating and managing the large, physically consistent simulation datasets required for training. To address this challenge, we present Agents4GEOS, an AI-agent framework built on the Model Context Protocol (MCP) that provides 52 domain-aware tools for natural-language-driven workflows with GEOS, an open-source multi-physics simulator. The agent facilitates input-file creation, mesh inspection, fluid-property computation, and result post-processing. Through human-curated skills and fresh-context subagents coordinated by an orchestrator, the system executes complex workflows, evaluates simulation outputs, diagnoses issues, and suggests improvements, grounding every quantity in actual computation. By automating routine tasks, Agents4GEOS allows domain experts to focus on the most challenging aspects of their work.

Figures

Figures reproduced from arXiv: 2607.18557 by Adriano M. A. C\^ortes, Alvaro L. G. A. Coutinho, Fernando A. Rochinha, Herv\'e Gross, Mauricio Araya-Polo, Roberto M. Velho.

Figure 1
Figure 1. Figure 1: Agents4GEOS reframes the task of simulation data generation as a natural-language dialogue. The user [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Layered architecture of Agents4GEOS. A natural-language request flows down through the agent and tool [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: End-to-end orchestration in Agents4GEOS for a simulation-creation request. The orchestrator routes the [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Plain English prompt for Agents4GEOS: A reservoir engineer asks Agents4GEOS to reproduce in GEOS the [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Agents4GEOS decision point: "The papers use an immiscible COz-brine system (no dissolution). How should I represent that in GEOS?". The input deck build followed the /geos workflow of section 2. The catalog knowledge base routed the request to the dead-oil model and selected an initial XML, a curated CO2-storage GEOS example, that connects the reservoir–well and hydrostatic equilibrium, which was then tran… view at source ↗
Figure 6
Figure 6. Figure 6: Initial gas saturation versus time graph produced by the subagent [PITH_FULL_IMAGE:figures/full_fig_p009_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The payoff of the reproduction session: gas saturation versus time at the three fixed observation points of [ [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Top: the permeability spatial field of the PUNQ-S3 benchmark. Bottom: a representation of the graph [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: GEOS solutions for the PUNQ-3D non-hysteresis case from [ [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: GEOS solutions for the PUNQ-3D hysteresis case from [ [PITH_FULL_IMAGE:figures/full_fig_p012_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: CO2 saturation at t = 100 years for the no hysteresis case obtained with (left) Plumecast GNN surrogate and (right) GEOS. hyperparameter selection, training management on GPU clusters, and uncertainty quantification. The aim is to reduce substantially the expertise required to build reliable surrogates for the many-query applications — history matching, optimization, and uncertainty quantification — that … view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

14 extracted references · 2 linked inside Pith

  1. [1]

    J. Guo, C. Park, D. Qian, T. J. R. Hughes, and W. K. Liu. Large language model-empowered next-generation computer-aided engineering.Computer Methods in Applied Mechanics and Engineering, 450:118591, 2026

    2026

  2. [2]

    Vishwasrao, F

    A. Vishwasrao, F. Giral, M. Golestanian, F. Tonti, A. A. Ramo, A. Lozano-Duran, S. L. Brunton, S. Hoyas, S. Clainche, H. Gomez, et al. Agentic exploration of pde spaces using latent foundation models for parameterized simulations.arXiv preprint arXiv:2604.09584, 2026. 13 A. M. A. Côrtes et al

    Pith/arXiv arXiv 2026

  3. [3]

    Jiang and G

    Q. Jiang and G. Karniadakis. Agenticsciml: Collaborative multi-agent systems for emergent discovery in scientific machine learning.npj Artificial Intelligence, 2026

    2026

  4. [4]

    Toscano, D

    J. Toscano, D. Chen, and G. Karniadakis. Athena: Agentic team for hierarchical evolutionary numerical algorithms. arXiv preprint arXiv:2512.03476, 2025

    Pith/arXiv arXiv 2025

  5. [5]

    Ramatullayev, S

    S. Ramatullayev, S. J. Su, N. El Droubi, M. Elfeel, K. Mansour, Z. Al Kindi, and A. Zand. Multi-agentic generative AI framework for accelerating field development planning. InProceedings of the IPTC Summit on AI for the Energy Industry, IPTC-25230-MS, Dubai, UAE, January 2026. International Petroleum Technology Conference

    2026

  6. [6]

    Onishi and Z

    T. Onishi and Z. Charpy. 24/7 simulation loops: How agentic AI keeps subsurface engineering moving. NVIDIA Technical Blog, April 2026

    2026

  7. [7]

    X. Ju, F. P. Hamon, G. Wen, R. Kanfar, M. Araya-Polo, and H. A. Tchelepi. Learning CO2 plume migration in faulted reservoirs with graph neural networks.Computers & Geosciences, 193:105711, 2024

    2024

  8. [8]

    R. S. Luna, T. H. Coelho, L. S. Neto, R. M. Velho, A. M. A Côrtes, R. N. Elias, A. G. Evsukoff, F. A. Rochinha, H. Gross, M. Araya-Polo, and A.L.G.A. Coutinho. Benchmarking advanced graph neural networks for co2 plume migration in complex geological formations. InSPE/AAPG/SEG Carbon, Capture, Utilization, and Storage Conference and Exhibition, page D011S0...

    2026

  9. [9]

    fluidflower

    H. Lu, L. Salo-Salgado, Y . Chou, E. Haghighat, and R. Juanes. Learning and inferring multiphase flow dynamics in porous media using scientific machine learning: Application to the "fluidflower" co2 injection experiment, 2026

    2026

  10. [10]

    R. R. Settgast, R. M. Aronson, J. R. Besset, A. Borio, Q. M. Bui, T. J. Byer, N. Castelletto, A. Citrain, B. C. Corbett, J. Corbett, et al. Geos: A performance portable multi-physics simulation framework for subsurface applications.Journal of Open Source Software, 9(LLNL-JRNL-864747), 2024

    2024

  11. [11]

    Saló-Salgado, R

    L. Saló-Salgado, R. Juanes, H. M. Nilsen, and O. Møyner. Three-dimensional simulation of geologic carbon dioxide sequestration using MRST.Advances in Geo-Energy Research, 13(2):73–89, 2024

    2024

  12. [12]

    Juanes, E

    R. Juanes, E. J. Spiteri, F. M. Orr, and M. J. Blunt. Impact of relative permeability hysteresis on geological CO 2 storage.Water Resources Research, 42(12):W12418, 2006

    2006

  13. [13]

    Pfaff, M

    T. Pfaff, M. Fortunato, A. Sanchez-Gonzalez, and P. W. Battaglia. Learning mesh-based simulation with graph networks. InInternational Conference on Learning Representations (ICLR), 2021

    2021

  14. [14]

    Y . Sun, X. Han, W. Zhang, Y . Pang, T. Wang, Y . Cao, Y . Huang, D. Song, et al. Agents’ last exam.arXiv preprint arXiv:2606.05405, 2026. 14

    arXiv 2026