{"id":"0d0e0fed-484e-42df-b850-2ac25ba0c663","arxiv_id":"2411.14192","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"LSE-MGN, a graph neural network with long- and short-edge message passing, predicts pore-scale gas/liquid evolution from experimental micro-CT data with roughly 9 to 10 percent surface-area error over short autoregressive rollouts.","lead":"Researchers trained a graph neural network on micro-CT scans of gas and water moving through rock, and it can predict how the fluids rearrange over the next few time steps. The paper is a proof of concept for learning pore-scale multiphase flow directly from experiment, with potential use in CO2 storage and hydrogen storage.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Without a persistence baseline or any out-of-distribution test, the low surface-area error does not demonstrate learned pore-scale physics; a near-identity predictor could match it.","rationale":"The reader's CONDITIONAL verdict is appropriate. My stress-test focuses on the evaluation protocol rather than dataset size per se. The absence of a persistence/trivial baseline is the most load-bearing weakness because it directly threatens the strongest claim: if the metric cannot distinguish learned dynamics from 'do nothing', no amount of architectural detail supports 'successfully captures the physics'. The model outputs probabilities, but surface-area error is computed on binary fluid data; the thresholding step is not specified. Also, the test intervals (t=221-229 and t=288-296) are temporally separated from training intervals (t=61-69 and t=198-206) but from the same experiment and same subvolume, so this is not evidence for generalizing across boundary conditions. I agree partially with the reader: representativeness of the subvolume is a concern, but the more immediate, testable issue is the missing baseline and lack of any held-out geometry/flow condition. Thus verdict should remain CONDITIONAL but with the condition explicitly including a persistence baseline and out-of-distribution evaluation.","tokens_in":7635,"tokens_out":3412,"duration_ms":35234,"concrete_test":"Implement a persistence baseline: for each test rollout, set the prediction at time t+1 equal to the ground-truth frame at time t, then compute the same surface-area error (Eq. 7) on the identical test frames. If the persistence error is within, say, 2 percentage points of the reported 8.81-9.88%, the improvement over doing nothing is not demonstrated. As a second check, train on the first 200 time steps and evaluate on the last 100 time steps of the same experiment, and, if possible, on a different subvolume or a different flow-rate experiment, to test true out-of-distribution generalisation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central generalization claim ('generalise well across varying boundary conditions', Section 1) is not tested. All training and test frames come from the same 50x80x50 subvolume of one carbonate core, one fluid pair, and one flow condition. The test set is not a different boundary condition; it is held-out time steps from the same experimental run. The only quantitative metric is the mean surface-area error (Eq. 7) with no baseline. Because the temporal spacing is 2 s and the flow is intermittent but locally quasi-static, a persistence forecast (predicting the previous frame) may achieve a similar surface-area error. If so, LSE-MGN could be learning a near-identity map on the gas/liquid field, and the reported 8.8-9.9% error would not indicate that any pore-scale physics has been learned. The paper also provides no error bars, no code/data release, and the training/test split is described inconsistently (40 vs 20 training frames; 16 vs 9 test frames), so the headline numbers cannot currently be checked against a baseline.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes LSE-MGN, a graph neural network that learns pore-scale two-phase (gas/liquid) flow directly from segmented micro-CT experimental images of a carbonate rock. The model represents pore voxels as nodes, uses a long-short-edge multi-scale message-passing architecture, and is trained with a soft Dice-BCE loss to autoregressively predict future gas/liquid distributions. The authors report a mean surface-area error of about 8.8-9.9% on held-out time steps from the same experimental sequence and claim that the model captures pore-scale physics and generalizes across boundary conditions.","tokens_in":7935,"tokens_out":2851,"duration_ms":27285,"significance":"Learning pore-scale multiphase flow from real experimental data rather than from idealized simulations is a worthwhile goal with clear relevance to CO2 storage, hydrogen storage, and fuel-cell applications. The paper introduces a sensible graph representation of the pore space and a multi-scale message-passing scheme that appears to enlarge receptive fields while reducing computational cost. The fact that the model is trained and evaluated on synchrotron micro-CT data, not synthetic data, is a strength. However, the quantitative evidence is currently limited: there is no comparison with any baseline (persistence, prior ML models, or numerical simulation), no error bars, and the generalization claim is not tested outside the single 50x80x50 subvolume and single flow condition. As a result, the paper demonstrates a promising architecture but does not yet substantiate the central claims of learning generalizable pore-scale physics.","major_comments":[{"comment":"The dataset split is reported inconsistently. Section 4 states the training set comprises 20 emerging-bubble and 20 oscillation time steps, and the test set includes 8 emerging-bubble and 8 oscillation steps. Table 1 and Appendix C instead state a 20-time-step training set and a 9-time-step test set. Table 2 reports testing error 0.0988 while Table 1 reports 0.0881 for the same configuration; these numbers cannot both correspond to the same split. This inconsistency directly affects the reproducibility of the headline results and must be resolved.","section":"Section 4 and Appendix C"},{"comment":"No baseline comparison is provided for the mean surface-area error. Because the flow is intermittent but locally quasi-static and the temporal resolution is 2 s, a persistence forecast (predicting the previous frame) may achieve a surface-area error of similar magnitude. Without such a baseline, the reported 8.8-9.9% error does not demonstrate that the model has learned pore-scale physics; it could be largely a near-identity map on the gas/liquid field. The paper should report the surface-area error of a persistence predictor and ideally also a simple voxel-wise majority-class baseline on the same test frames.","section":"Section 4, Eq. (7)"},{"comment":"The claim that the model 'can effectively capture complex fluid behaviors and generalise well across varying boundary conditions' is not tested. All training and test data come from the same 50 x 80 x 50 subvolume of one carbonate core, one fluid pair, and one flow condition; the test set consists of held-out time steps from the same experimental run. To support the generalization claim, the authors should either provide out-of-distribution experiments (different subvolumes, flow rates, or fluid pairs) or substantially temper the claim to 'generalizes to unseen time steps within the same experiment.'","section":"Abstract and Section 1"},{"comment":"No error bars or repeated-seed experiments are reported. The differences between configurations in Table 1 (e.g., 8.81% vs 9.05% vs 10.10%) are likely within run-to-run variability for a stochastic optimizer, so the model selection F6C6F6 is not statistically justified. The paper should report the mean and standard deviation over at least a few training seeds, and ideally show that the chosen configuration's advantage over F7C4F7 persists.","section":"Table 1 and Section 3"},{"comment":"The surface-area error is a global quantity that can be insensitive to spatial mislocalization of fluid clusters. Two predictions with identical surface area can place gas in completely different pores. The paper should supplement this metric with a voxel-wise metric such as intersection-over-union or accuracy, and ideally report errors separately for the bubble-emergence and oscillation test sets, since these are distinct scenarios.","section":"Section 4, Eq. (6)-(7)"}],"minor_comments":[{"comment":"The coarse-message-passing equation uses the indicator I(|e'_{ij}| = sqrt(3)), but e'_{ij} is a latent edge embedding, not a distance. The filtering should apply to the original edge length (which is one of the input edge attributes), not to the updated embedding. Please clarify the notation.","section":"Section 3, Eq. (4)"},{"comment":"The symbols N and n in the soft Dice BCE loss are not defined. Presumably N is the total number of nodes and n is the number of gas voxels in the ground truth, but this should be stated explicitly.","section":"Section 3, Eq. (5)"},{"comment":"The text says micro-CT imaging provides 'nanometer-scale spatial resolution,' but the stated resolution is 2.75 µm (micrometres), not nanometers. This should be corrected.","section":"Section 2"},{"comment":"The caption refers to 'the testing set with emerging bubbles' and shows time steps t=221 to 229, which is 9 frames, while Section 4 states the test set contains 8 emerging-bubble steps. Please clarify whether the displayed interval is a subset or whether the test set description is inaccurate.","section":"Figure 2 caption"},{"comment":"The surface-area metric computed via Marching Cubes depends on the voxel discretization. This is acceptable since the same algorithm is applied to predictions and ground truth, but the units should be stated (e.g., voxel surface area, normalized by the domain size).","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript appears to be a preprint version of a conference submission (NeurIPS 2024), and the journal version will need substantial strengthening to meet the standards of a full research article. The main weaknesses are the absent baseline comparisons and the untested generalization claim. Both are fixable within the scope of the paper, so I recommend major revision rather than rejection. The authors should also be encouraged to release code and data, as the reproducibility of the reported numbers currently cannot be checked given the inconsistent split descriptions."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know: this is a reasonable proof-of-concept, not a demonstrated general model. The new bit is applying GraphNet-style learning directly to experimental micro-CT data, and the LSE-MGN long-short-edge message passing is a sensible tweak that avoids precomputed coarse graphs. On its own terms it works: autoregressive rollouts track bubble emergence and oscillation with about 9% surface-area error on held-out frames from the same experiment. That is a genuine but narrow result.\n\nWhat's good: using real intermittent-flow data instead of simulation is the right call, and the architecture choices are clearly explained. The loss comparison in Appendix C is useful. The paper is honest about the difficulty of the problem.\n\nWhere it falls short: the central claim of generalising across boundary conditions is unsupported. Every test frame comes from the same 50x80x50 subvolume of one carbonate core, one fluid pair, one flow condition. Held-out time steps are not a different boundary condition. The stress-test worry is fair: with 2-second spacing and locally quasi-static flow, a persistence baseline that just copies the previous frame might hit similar surface-area error. Without that baseline, the 9% number doesn't prove the model learned physics. The paper also reports inconsistent split sizes (20 vs 40 training frames, 8 vs 9 test frames), gives no error bars or repeated seeds, and releases no code or data. The surface-area metric is global; it can miss local mispredictions.\n\nNone of this kills the core idea. The architecture is plausible, the training on experimental data is novel, and the visual rollouts look credible. But the paper needs major revision before its claims are accepted: add a persistence baseline and a simulation baseline, test on a different subvolume or flow rate, fix the split reporting, and release code/data.\n\nWho should read it: people working on ML surrogates for porous media, especially CO2/H2 storage. It deserves peer review, not a desk reject, but a careful referee will send it back for a substantive revision.","headline":"Promising proof-of-concept for learning pore-scale multiphase flow from micro-CT data, but the generalization claim is untested and the evaluation lacks baselines.","tokens_in":8361,"tokens_out":1855,"would_cite":false,"duration_ms":17643,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A graph neural network trained on micro-CT images can predict how gas and liquid move through rock pores.","keywords":["graph neural network","pore-scale multiphase flow","micro-CT imaging","porous media","autoregressive prediction","intermittent flow","mesh graph networks","CO2 storage"],"falsifier":"Take the trained LSE-MGN and run autoregressive rollout on a different subvolume of the same core, or on a different rock sample with a different pore structure; if the mean surface-area error rises well above the reported 8-10 percent range, the claim that the model learned general pore-scale physics from this experimental dataset would be contradicted.","tokens_in":7402,"feed_emoji":"💧","tokens_out":4666,"duration_ms":39868,"temperature":0.7,"pith_summary":"The paper sets out to show that a graph neural network can learn the pore-scale physics of multiphase flow directly from real micro-CT experimental images, without solving the Navier-Stokes equations. Its proposed model, LSE-MGN, represents the pore space as a graph and autoregressively predicts whether each pore voxel holds gas or liquid at the next time step. The authors report that the model captures both bubble emergence and minor oscillations in a carbonate rock sample, with mean surface-area error around 8.8 to 9.9 percent on held-out frames. If this holds, it would offer a fast, accurate route to modelling flows relevant to CO2 storage, hydrogen storage, and fuel cells.","feed_headline":"Graph network learns pore-scale flow from real rock scans","feed_subtitle":"Trained on micro-CT images, it predicts gas and liquid movement with about 9 percent surface-area error.","key_machinery":"The load-bearing mechanism is a graph representation of the segmented pore space in which each node is a pore voxel with features for gas/liquid state, interface status, coordinates, and local gas fraction, and edges connect nodes within a radius of $\\sqrt{3}$ voxel units. The LSE-MGN architecture uses an encoder-processor-decoder layout, but replaces single-scale message passing with a long-short-edge scheme: all edges propagate in fine processors, then only 'long' edges (length $\\sqrt{3}$) propagate in coarse processors, letting pivotal nodes exchange information across distant pores, then all edges propagate again. This expands each node's receptive field while keeping the coarse graph under 30 percent of the original edges. The training objective is the soft Dice BCE loss, which balances per-voxel accuracy against the class imbalance between gas and liquid.","core_discovery":"On the paper's own terms, the central discovery is that a graph network trained on only 40 time steps of experimental micro-CT data—20 of an emerging bubble and 20 of minor oscillation—can autoregressively generate future gas/liquid distributions that match the measured pore-scale flow. The model tracks the constantly rearranging flow pathways characteristic of intermittent flow, something the authors say current numerical models cannot capture. The best configuration, F6C6F6 with six fine, six coarse, and six fine message-passing layers, reaches a mean surface-area error of 8.81 percent on a 9-step validation set, and the soft Dice BCE loss achieves 9.88 percent on the 16-step test set. The authors take this as evidence that the model successfully captures the physics from high-resolution experimental data and generalises across the boundary conditions present in the experiment.","pith_inferences":["The paper's evidence for 'generalisation across varying boundary conditions' is confined to two flow regimes inside one subvolume; a stronger test would train on several subvolumes or cores and measure transfer performance, which the authors do not report.","The surface-area error metric may not detect errors in gas-phase connectivity or trapping; adding topological metrics such as the Euler characteristic would make evaluation stricter and might change the ranking of loss functions.","A natural follow-up experiment is to train on multiple intermittent-flow subvolumes and test on a steady-flow subvolume, which would clarify whether the model learns intermittent dynamics specifically or a more general pore-scale flow rule.","If the model were instead trained on simulated data and fine-tuned on these 40 experimental frames, the comparison would reveal how much of the learned behaviour comes from the experimental data itself."],"forward_implications":["If the method holds, pore-scale flow prediction becomes possible at micro-CT resolution without solving the Navier-Stokes equations, enabling fast screening of injection strategies for CO2 storage.","The graph-based representation avoids the grid limitations of convolutional networks and can in principle scale to larger rock volumes through section-based training.","Autoregressive rollout from a single initial state means long flow sequences can be generated cheaply, potentially replacing expensive pore-network model calibrations.","The same architecture could be retrained on other fluid pairs or rock types once experimental imaging data are available."],"supporting_citations":[{"why":"Supplies the micro-CT experimental dataset of nitrogen and brine flow through carbonate rock, including the 50×80×50 subvolume and 300 time steps used for training and testing.","marker":"[8]"},{"why":"Provides the MeshGraphNet architecture that LSE-MGN extends with long-short-edge message passing.","marker":"[14]"},{"why":"Introduces the encoder-processor-decoder graph-network simulator design and the idea of zero-shot generalisation across boundary conditions.","marker":"[15]"},{"why":"Offers the multi-scale message-passing strategy that motivates splitting edges into long and short categories to widen receptive fields.","marker":"[18]"},{"why":"Describes the segmentation workflow that converts raw X-ray images into the gas/liquid/rock labels used as node states.","marker":"[17]"},{"why":"Defines the soft Dice BCE loss that the paper selects as the best-performing training objective.","marker":"[19]"}],"fun_headline_variants":["Graph nets learn pore-scale flow from experiments, not simulations","AI predicts gas-liquid movement in rock pores from real data","Graph network captures observed multiphase flow dynamics in pores","Autoregressive graph model matches experimental pore-scale fluid flow"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The paper assumes that a single hand-picked 50 by 80 by 50 voxel subvolume of one carbonate core, with only 40 training time steps, contains enough variety of pore-scale flow behaviour to teach a model that generalises to other geometries, flow rates, and boundary conditions.","fun_headline_variants_meta":{"raw":{"variants":["Graph nets learn pore-scale flow from experiments, not simulations","AI predicts gas-liquid movement in rock pores from real data","Graph network captures observed multiphase flow dynamics in pores","Autoregressive graph model matches experimental pore-scale fluid flow"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000238,"raw_usage":{"total_tokens":1472,"prompt_tokens":868,"completion_tokens":604,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":484,"completion_tokens_details":{"reasoning_tokens":537}},"tokens_in":484,"tokens_out":604,"duration_ms":5713,"temperature":1.0,"reasoning_tokens":537,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:25:46.595062+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the trained LSE-MGN and run autoregressive rollout on a different subvolume of the same core, or on a different rock sample with a different pore structure; if the mean surface-area error rises well above the reported 8-10 percent range, the claim that the model learned general pore-scale physics from this experimental dataset would be contradicted.","supporting_citations":[{"cited_title":"Real-time imaging reveals distinct pore-scale dynamics during transient and equilibrium subsurface multiphase flow","cited_arxiv_id":null,"evidence_quote":"Supplies the micro-CT experimental dataset of nitrogen and brine flow through carbonate rock, including the 50×80×50 subvolume and 300 time steps used for training and testing."},{"cited_title":"Python workflow for segmenting multiphase flow in porous rocks","cited_arxiv_id":null,"evidence_quote":"Describes the segmentation workflow that converts raw X-ray images into the gas/liquid/rock labels used as node states."},{"cited_title":"On the optimal combi- nation of cross-entropy and soft dice losses for lesion segmentation with out-of-distribution robustness","cited_arxiv_id":null,"evidence_quote":"Defines the soft Dice BCE loss that the paper selects as the best-performing training objective."}],"review_version":1}