{"id":"37c8acf0-015b-464b-82dc-c02270ac22a2","arxiv_id":"1908.07841","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A topology-aware lifted Wasserstein metric ranks viscous finger simulations against an acquired ground truth more accurately than overlap, persistence-only, or geometry-only baselines.","lead":"The paper presents a topological data analysis framework that automatically ranks viscous fingering simulations against an X-ray reference image of a real flow experiment. It lets engineers quickly discard unrealistic simulation runs and inspect the ones whose finger geometry and timing best match the acquisition.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Best-metric claim is under-validated: \\hat W2 parameters and the saturation threshold are expert-tuned (Secs. 3.1-3.2) yet evaluated on a single expert ranking (Sec. 4.1), with no sensitivity analysis or error bars, so tau=0.84 may reflect overfitting.","rationale":"The paper is a useful application study, and I do not question the in-situ implementation, the documented performance gains, or the qualitative expert feedback. The load-bearing issue is that the central quantitative claim, namely that \\hat W2 manages to combine the advantages of both EMD and W2, is supported only by a single, expert-tuned evaluation. Even if the persistence representation is accepted, the threshold and lifting coefficients were chosen with the same kind of expert judgment later used as the reference, so the reported Kendall coefficient does not separate the method's intrinsic merit from the tuning expert's preferences. A threshold and coefficient sweep would settle this directly. The reader's weakest assumption about finger-tip representation is related but downstream: representation failure would invalidate all metrics, whereas the tuning concern targets the specific claim that \\hat W2 is best. I therefore partially agree with the reader and keep the same conditional verdict.","tokens_in":20424,"tokens_out":7393,"duration_ms":84488,"concrete_test":"Recompute Table 2 for a grid of threshold values w in {0.08, 0.10, 0.12, 0.14, 0.16} and lifting coefficients around the chosen beta_x=10/gamma and alpha_x=alpha_y=1/rho, keeping all other pipeline choices fixed. If \\hat W2's top-25 Kendall tau drops below 0.6 anywhere in that neighborhood, the reported superiority is not robust to the expert-set meta-parameters; if it remains above 0.6, the overfitting concern is substantially weakened.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.2 fixes the lifted-Wasserstein coefficients beta_x=10/gamma, beta_y=0, alpha_x=alpha_y=1/rho based on discussions with experts, and Sec. 3.1 fixes the saturation threshold w=0.12 with expert input; Sec. 4.1 then uses expert rankings as the ground truth against which all six metrics are scored. The claim that \\hat W2 achieves the best overall Kendall coefficients (Sec. 4.3) is therefore not an independent comparison: the same kind of expert judgment is used both to set the metric's meta-parameters and feature representation and to label the evaluation. The study is one realization, namely one slab, one expert ranking, eight matched time steps, and 200 runs, with no confidence intervals, bootstrap, or cross-validation reported. Additionally, Table 2's top-25 coefficients are computed on the subset each method itself selects, which can inflate apparent agreement. The Conclusion's own statement that the meta-parameters would likely need to be adjusted and that their automatic optimization is future work underscores that tau=0.84 has not been shown robust outside the tuned setting.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a topological data analysis (TDA) framework for ranking members of a viscous finger simulation ensemble against an X-rayacquired ground truth. Fingers are represented as persistence pairs of the x-coordinate scalar field on a thresholded sublevel set of water saturation, and several time-integrated distances between the resulting persistence diagrams are introduced: the Earth mover's distance (EMD), the 2-Wasserstein distance (W2), its geometrically lifted version (W2hat), and velocity-oriented variants based on intra-run fingerprint tracking. The distances are evaluated by comparing the induced rankings with an expert reference ranking using Kendall's tau, on 200 simulation runs and 8 matched time steps from a single slab experiment. The paper reports that W2hat achieves the best ranking agreement (tau = 0.84 on the top 25 runs), and documents an in-situ implementation that reduces data movement by five orders of magnitude and speeds up per-time-step analysis by a factor of 2.3.","tokens_in":20728,"tokens_out":3752,"duration_ms":40438,"significance":"If the ranking-quality claim were robustly established, this would be a practically valuable application of TDA to reservoir engineering, combining a domain-specific feature representation, a lightweight visual interface, and an in-situ deployment. The paper's strengths include a clearly specified pipeline built on the open-source Topology ToolKit, a complete case study with expert feedback, and a direct comparison of several established and novel metrics. The main quantitative claim, however, rests on a single acquisition, a single expert ranking, and expert-tuned parameters, so the significance of the measured superiority of W2hat is not yet firmly established.","major_comments":[{"comment":"The evaluation of the proposed metrics is circular in an important sense: the saturation threshold w=0.12 (Sec. 3.1) and the lifting coefficients beta_x=10/gamma, alpha_x=alpha_y=1/rho (Sec. 3.2) are set based on discussions with experts, and the same type of expert judgment is then used to build the reference ranking against which all metrics are scored (Sec. 4.1, Table 2). Consequently, the Kendall coefficient tau=0.84 for W2hat in Sec. 4.3 is not an independent measure of ranking quality. The paper should provide evidence that the result is robust to reasonable perturbations of these parameters (e.g., a sensitivity analysis over w and the lifting coefficients), or validate the metric on an independent expert ranking or a held-out subset of runs. Without such evidence, the claim that W2hat 'manages to combine the advantages of both EMD and W2' is not fully supported.","section":"Sec. 3.1, Sec. 3.2, Sec. 4.1, Sec. 4.3"},{"comment":"The top-50 and top-25 Kendall coefficients are computed on the subset of runs that each method itself ranks at the top. This selection procedure biases the comparison in favor of methods whose top selections happen to coincide with the expert's ranking and makes cross-method comparison difficult. For example, a method that places a different set of runs at the top is evaluated on a different set of items. The paper should report coefficients on fixed subsets (e.g., the top 50 runs according to the expert reference) or use a rank-correlation measure that is robust to censoring, along with permutation-based or bootstrap confidence intervals.","section":"Sec. 4.3, Table 2"},{"comment":"The central feature representation assumes that finger tips correspond to local maxima of the x-coordinate field on the thresholded sublevel set F = f^{-1}_{-infty}(0.12) and that the persistence of each pair equals finger length. This assumption is asserted but not validated against the manually segmented reference finger geometry FA described in Sec. 4.1. If the threshold or the x-coordinate field fails to capture finger tips (e.g., for diffuse fronts or merging fingers), all downstream metrics inherit that failure. The paper should provide quantitative or at least systematic qualitative evidence that the extracted persistence pairs correspond to the fingers that experts identify in the acquisition and in the simulations.","section":"Sec. 3.1"},{"comment":"The study is a single realization: one slab, one acquisition, one expert ranking, and eight matched time steps, with no confidence intervals or uncertainty quantification on the Kendall coefficients. The conclusion itself acknowledges that the meta-parameters 'would likely need to be adjusted' in other scenarios and that automatic optimization is future work. Given this, the paper should temper the claim of 'quantitative superiority' of W2hat and clearly state that the result is a proof-of-concept on one case study rather than a demonstrated general property of the metric. At minimum, a bootstrap over the 200 runs or over ranking pairs should be added to show the stability of the reported coefficients.","section":"Sec. 4.3, Sec. 5"}],"minor_comments":[{"comment":"The overlap O(At,St) is defined as a ratio of 'volumes,' but the domain is 2D and the data are images; 'area' or 'measure' would be more precise.","section":"Sec. 4.1"},{"comment":"The notation At is used both for the acquired time step and for the acquisition sequence; this can be confusing in equations such as d_qW2(At,At+1). Please distinguish the time index from the data object.","section":"Sec. 3.2"},{"comment":"In the definition of d_qW2, the roles of nSt and nAt in the fractions 1/nSt and 1/nAt should be clarified, and the treatment of the last time step (where t+1 may not exist) should be stated explicitly.","section":"Sec. 3.2"},{"comment":"The abstract says 'Extensive experiments,' but the quantitative evaluation is based on a single case study with 200 runs and one expert ranking. Consider rephrasing to 'a complete case study' or 'detailed experiments on one acquisition.'","section":"Abstract and Sec. 4"}],"recommendation":"major_revision","confidential_remarks":"The paper is a reasonable application-oriented contribution, and the in-situ implementation and web interface are useful. The main issue is that the best-metric claim is under-validated because of the circularity between expert-tuned parameters and the expert-based reference ranking, and because of the small number of independent evaluations. These concerns are addressable with additional analyses (sensitivity studies, bootstrap, and fixed-subset rank correlations), so I recommend major revision rather than rejection. The manuscript's claims should be toned down until such evidence is provided."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know two things before reading. First, this is a genuine application of TDA to reservoir engineering, and it includes a working in-situ pipeline with real performance numbers. Second, the headline claim—that the lifted Wasserstein metric \\hat W2 ranks viscous finger simulations best—is not as robust as the paper suggests: the metric's meta-parameters and the saturation threshold were set with expert input, and the evaluation uses expert judgment as ground truth.\n\nWhat's actually new: the feature representation is the x-coordinate scalar field on the thresholded water-saturation sublevel set, so maxima become finger tips and persistence becomes finger length. That is a sensible domain adaptation, and it is not done in the prior viscous-finger TDA papers. They also introduce time-integrated and velocity-aware distances between persistence diagrams. The in-situ implementation is a real contribution: it cuts data movement by five orders of magnitude and roughly halves analysis time. The evaluation is internally consistent, and \\hat W2 does achieve the best Kendall coefficient (0.84 on the top 25 runs).\n\nThe stress-test note is on target. The coefficients beta_x = 10/gamma, alpha_x = alpha_y = 1/rho, and the threshold w = 0.12 are set \"based on discussions with experts\" (Secs. 3.1 and 3.2), and the same kind of expert judgment produces the reference ranking in Sec. 4.1. That is not a fatal circularity—the tuning is coarse and not directly optimized against the Kendall score—but it means tau = 0.84 could partly reflect parameter choice. The evidence is also one slab, one expert ranking, eight matched time steps, 200 runs, and no confidence intervals, bootstrap, or cross-validation. The top-25 coefficients are computed on the subset each method itself selects, which can inflate apparent agreement. To the paper's credit, the conclusion explicitly says the meta-parameters would likely need adjustment and that automatic optimization is future work; that is honest but it also undercuts the strength of the current claim.\n\nWho should read this: TDA and visualization people interested in a detailed application case study, and reservoir engineers who want an automated ranking tool. It deserves a serious referee, not a desk reject. I would ask for a sensitivity analysis on the meta-parameters, and either a second independent ranking or a clear statement that the result is a proof of concept. With those additions, the case would be solid.","headline":"Solid, well-scoped application paper whose best-metric claim needs a sensitivity analysis before it can be taken as more than a single tuned case study.","tokens_in":21204,"tokens_out":1929,"would_cite":false,"duration_ms":20963,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A lifted Wasserstein distance on persistence diagrams of finger tips ranks viscous-finger simulation runs against an X-ray ground truth with rank correlation 0.84 on the top 25 runs, better than overlap, EMD, and plain Wasserstein.","keywords":["viscous fingering","persistence diagrams","topological data analysis","Wasserstein distance","optimal transport","ensemble simulation ranking","in-situ analysis","porous media flow"],"falsifier":"Run the same pipeline on the 200 simulations with the fixed threshold replaced by an automatic per-time-step segmentation, and check whether $\\hat{W}_2$'s rank correlation on the top 25 still exceeds EMD's 0.70; if it does not, the reported ranking quality is tied to the hand-set threshold. A complementary check is to have a second independent panel of experts re-rank the same runs and measure the panel's rank correlation with the original reference: if human-to-human agreement is below 0.84, the metric's agreement with one expert panel cannot be read as superior ranking ability.","tokens_in":20268,"feed_emoji":"💧","tokens_out":15400,"duration_ms":138447,"temperature":0.7,"pith_summary":"This paper argues that viscous fingers in porous-media flow can be represented by persistence diagrams, topological summaries of when features appear and disappear in a scalar field, and that a geometrically lifted Wasserstein distance (an optimal-transport matching between diagrams) ranks an ensemble of simulations against an X-ray acquisition more faithfully than overlap, unweighted Wasserstein, or geometry-only optimal transport. The practical payoff is an automatic screening tool: instead of visually inspecting hundreds of runs, an engineer can take the top of the ranking and discard runs that are too slow, too fast, or too diffuse. In a 200-run case study with eight acquired snapshots, the lifted metric $\\hat{W}_2$ reaches a rank-correlation coefficient of 0.84 on the 25 best runs identified by each method, ahead of plain Wasserstein (0.29), the Earth mover's distance (0.70), and overlap (0.13). The paper further claims the ranking can be computed in-situ during simulation, reducing data movement by about five orders of magnitude.","feed_headline":"Metric that pairs finger tips ranks viscous runs like an expert","feed_subtitle":"Lifting persistence diagrams with finger-tip positions beats overlap, EMD, and plain Wasserstein in matching expert rankings.","key_machinery":"The carrying mechanism is the persistence diagram of the $x$-coordinate scalar field restricted to the thresholded sublevel set $F = f^{-1}_{-\\infty}(0.12)$. On $F$, a local maximum of $x$ is a finger tip and its paired saddle has persistence equal to finger length, so the diagram records the number of fingers, how far each has advanced, and how prominent it is, in a way that is insensitive to small noise. Distances between a simulation diagram and the acquisition diagram use the lifted point-wise cost $\\hat{d}_p$ of Eq. (6), with weight $\\beta_x = 10/\\gamma$ on the $x$-positions of tips, $\\beta_y = 0$, and $\\alpha_x = \\alpha_y = 1/\\rho$, and these per-time-step costs are accumulated with an $\\ell^2$ norm to form the ranking distance $d_{\\hat{W}_2}$. The lifting is what lets the matching sacrifice a small persistence difference to bind a fingertip to a geometrically close fingertip, which the paper shows is the step that recovers the expert-preferred association.","core_discovery":"The central claim is that the essential information for judging a viscous-fingering simulation lies not in the pixel geometry of the saturation field but in the number, progress, and prominence of finger tips, and that this information is carried by persistence pairs of the $x$-coordinate field on the thresholded finger set $F = f^{-1}_{-\\infty}(0.12)$. Comparing these diagrams with the lifted Wasserstein metric $\\hat{W}_2$ yields a ranking that agrees with expert judgment better than each of its ingredients: it reaches $\\tau = 0.84$ on the top 25 runs according to each method, while $W_2$ reaches 0.29, EMD 0.70, and overlap 0.13. The paper interprets this as $\\hat{W}_2$ combining the persistence sensitivity of $W_2$ with the tip-position sensitivity of EMD rather than being a simple interpolation. It also documents that velocity-oriented metrics based on distances traveled by fingers fail, because they reward slow diffuse runs or match smooth slow fronts to fast thin fingers.","pith_inferences":["Editorial inference: if the top-25 agreement survives a held-out acquisition, the same tip-weighted persistence matching should transfer to other unstable interface problems where finger-tip advancement is the physically decisive quantity, such as forced imbibition in narrow cells or miscible displacements; the paper does not test this transfer.","Editorial inference: because the threshold 0.12 and the lifting coefficients are fixed by expert discussion, the reported superiority could be tuned to the validation set; recomputing the ranking with an automatic per-image threshold would test whether the gap over EMD and $W_2$ persists.","Editorial inference: adding finger volume or merging events to the persistence representation is the paper's own suggested remedy for the diffuse runs that still enter the top 50 of $\\hat{W}_2$; a concrete test would compare rankings with and without this augmentation on the same ensemble.","Editorial inference: combining $d_{\\hat{W}_2}$ with production and pressure mismatches during history matching may expose the relative-permeability regime that finger geometry alone does not reveal."],"forward_implications":["An ensemble can be ranked automatically at run time, so engineers can focus manual inspection on the top-ranked simulations and discard implausible ones before history matching.","The same per-time-step distance accumulation works without storing scalar fields to disk, cutting analysis time by a factor of 2.3 and data movement by five orders of magnitude in the documented case study.","The framework is not restricted to two dimensions or to one flow model: the pipeline transfers to 3D and to other simulators, with the saturation threshold and lifting coefficients needing re-tuning.","The absence of a clear regime in relative-permeability space among the best runs means the metric ranks outcomes but does not, by itself, identify which input parameters produce the most plausible fingers."],"supporting_citations":[{"why":"Supplies the persistence diagram formalism and the critical-point pairing rule that defines finger features.","marker":"[32]"},{"why":"Introduces the lifted Wasserstein matching and the exact assignment solver that the paper's $\\hat{W}_2$ metric reuses.","marker":"[92]"},{"why":"Provides the task-based merge-tree algorithm used to extract persistence diagrams during simulation.","marker":"[42]"},{"why":"Defines the Earth mover's distance that serves as the geometry-only baseline in the ranking comparison.","marker":"[57]"},{"why":"Documents the X-ray slab acquisition protocol that generated the ground-truth finger images and the expert-validated scenario.","marker":"[29]"},{"why":"Gives the stability result for persistence diagrams that underlies the claim that low-persistence pairs can be treated as noise.","marker":"[19]"},{"why":"Documents the software library that implements the persistence and matching routines used in the experiments.","marker":"[100]"}],"fun_headline_variants":["Topology-aware metric ranks viscous finger simulations like experts","Lifted Wasserstein metric beats EMD and plain Wasserstein on viscous fingers","Persistence diagrams with finger-tip positions improve simulation ranking","New matching metric aligns simulation rankings with expert judgment","Finger tips and persistence lift simulation ranking to expert level"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The ranking stands or falls on the assumption that every relevant finger appears as one persistence pair of the $x$-coordinate field on the thresholded set at the fixed saturation value 0.12; a finger whose tip is not a clean local maximum of $x$ on $F$ is invisible to all of the compared metrics.","fun_headline_variants_meta":{"raw":{"variants":["Topology-aware metric ranks viscous finger simulations like experts","Lifted Wasserstein metric beats EMD and plain Wasserstein on viscous fingers","Persistence diagrams with finger-tip positions improve simulation ranking","New matching metric aligns simulation rankings with expert judgment","Finger tips and persistence lift simulation ranking to expert level"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000268,"raw_usage":{"total_tokens":1621,"prompt_tokens":952,"completion_tokens":669,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":568,"completion_tokens_details":{"reasoning_tokens":587}},"tokens_in":568,"tokens_out":669,"duration_ms":7276,"temperature":1.0,"reasoning_tokens":587,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T12:23:23.821702+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same pipeline on the 200 simulations with the fixed threshold replaced by an automatic per-time-step segmentation, and check whether $\\hat{W}_2$'s rank correlation on the top 25 still exceeds EMD's 0.70; if it does not, the reported ranking quality is tied to the hand-set threshold. A complementary check is to have a second independent panel of experts re-rank the same runs and measure the panel's rank correlation with the original reference: if human-to-human agreement is below 0.84, the metric's agreement with one expert panel cannot be read as superior ranking ability.","supporting_citations":[{"cited_title":"Edelsbrunner and J","cited_arxiv_id":null,"evidence_quote":"Supplies the persistence diagram formalism and the critical-point pairing rule that defines finger features."},{"cited_title":"Soler, M","cited_arxiv_id":null,"evidence_quote":"Introduces the lifted Wasserstein matching and the exact assignment solver that the paper's $\\hat{W}_2$ metric reuses."},{"cited_title":"Gueunet, P","cited_arxiv_id":null,"evidence_quote":"Provides the task-based merge-tree algorithm used to extract persistence diagrams during simulation."},{"cited_title":"Levina and P","cited_arxiv_id":null,"evidence_quote":"Defines the Earth mover's distance that serves as the geometry-only baseline in the ranking comparison."},{"cited_title":"de Loubens, G","cited_arxiv_id":null,"evidence_quote":"Documents the X-ray slab acquisition protocol that generated the ground-truth finger images and the expert-validated scenario."},{"cited_title":"Tierny, G","cited_arxiv_id":null,"evidence_quote":"Documents the software library that implements the persistence and matching routines used in the experiments."}],"review_version":1}