{"id":"62ecb658-9ccb-4ebf-8524-0194b4196ca3","arxiv_id":"2607.16587","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"An automated SAM2-based pipeline measures droplet flame diameter in microgravity combustion videos with about 3.1% average error versus manual reference and a 229x speedup.","lead":"This paper combines an off-the-shelf AI image segmenter (SAM2) with automatic point picking and a robust circle fit to measure flame diameters in microgravity droplet-combustion videos. Tested on about 19,500 images from three fuels, the automated tool agrees with manual human measurements to roughly 97% and runs about 229 times faster.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Accuracy claim is validated only against the same group's manual reference (8.5% uncertainty), so 96.9% agreement is not independent ground truth; independent validation is needed.","rationale":"The reader's weakest assumption identifies the manual-reference ground truth as the critical unvalidated link. The paper's headline numbers (96.9%, 3.1%) are agreement with this same-group reference, not with an independently established flame diameter. This is load-bearing: if the manual reference carries systematic bias in interpreting the blurred luminous boundary, the automated method could share that bias and the claimed 'accuracy' would be illusory. The uncertainty analysis in Section 3.7 does not resolve this because u_ref=0.085 dominates the combined uncertainty, essentially folding the unvalidated reference into the claimed metrological characterization. My concrete test of synthetic images with known diameters would settle whether the method is truly accurate. The conditionality expressed by the reader is appropriate; I do not see a basis to change the verdict, but I also do not see a reason to accept it unconditionally without the proposed independent check.","tokens_in":13512,"tokens_out":7943,"duration_ms":86460,"concrete_test":"Generate 500 synthetic flame images with known ground-truth diameters, incorporating realistic blur, soot tails, reflections, and drift; run the exact pipeline (with all hyperparameters, e.g., V>25, S>15, R+G+B>40, shrink 0.9, RANSAC threshold tau) and compare recovered diameters to the known values. If the bias exceeds 3% or varies by more than 1% when the hand-set thresholds are varied over plausible ranges, the 96.9% figure is not a true accuracy and the method needs an independent reference.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central accuracy claim (96.9% mean relative agreement, MAPE 3.1%) is computed exclusively against the manual-reference (m-measurement) flame diameters from the authors' prior work [5], which carry a stated 8.5% uncertainty and were produced by the same group using Image-Pro ellipse fitting. Because the auto-prompt parameters (HSV thresholds, shrink factor 0.9, RANSAC parameters in Section 3.1) were likely chosen with knowledge of these reference values, the reported agreement may reflect tuning to reproduce that specific manual interpretation rather than true flame-diameter accuracy. Section 3.7's combined uncertainty (8.54%) is dominated by the manual-reference term u_ref=0.085, so it does not independently validate the automated method. Without an independent ground truth, the claim of a 'metrologically characterized' replacement for manual measurement is not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents a fully automated image-analysis workflow for measuring droplet flame diameter in microgravity combustion videos. The pipeline combines automatic prompt-point generation with SAM2 video segmentation, RANSAC-based circle fitting, and temporal tracking via SAM2's memory mechanism. The authors validate the method on 19,537 flame images from ISS FLEX-2 experiments covering n-heptane, n-decane, and n-octane, reporting a mean relative agreement of 96.9% with manual-reference measurements, an 8.54% combined standard uncertainty, and an approximately 229x speed-up over manual measurement. The central claim is that this workflow provides a reproducible, fully automated, and metrologically characterized replacement for manual flame-diameter extraction.","tokens_in":13773,"tokens_out":3270,"duration_ms":38064,"significance":"If the claims are substantiated, this is a useful contribution to combustion diagnostics and to the broader use of foundation segmentation models in quantitative image metrology. The work draws on a large publicly available dataset, avoids fine-tuning SAM2, and tackles real challenges (soot tails, corona effects, droplet drift) with domain-aware strategies. The reported efficiency gain is substantial, and the method appears to generalize across three fuels. However, the significance is conditional: the headline accuracy is measured only against a manual reference produced by the same research group, and several key parameters were selected using that same reference. The paper does not yet provide independent evidence that the method measures the true flame diameter rather than reproducing the biases of the manual reference.","major_comments":[{"comment":"The validation ground truth is not independent. The manual-reference measurements m-measurements from Ref. [5] are generated by the same group and carry a stated 8.5% uncertainty. Eq. (8) adds u_ref=0.085 directly into the combined uncertainty, so the reported 8.54% is dominated by the reference uncertainty and does not characterize the automated method itself. Because the HSV thresholds, shrink factor, RANSAC parameters, and prompt-point count were selected by comparing to these same m-measurements in Sections 3.2–3.4, the 96.9% agreement is plausibly tuned to reproduce that specific manual interpretation. Please provide an independent validation—for example, on synthetic images with known flame diameters, or against multi-annotator manual measurements with inter-observer variability—and/or reframe the claim as 'agreement with manual reference' rather than absolute accuracy. The abstrac","section":"Section 3.1, 3.7; Eqs. (8)"},{"comment":"The reported accuracy numbers are inconsistent. Section 3.1 reports 98.19% for RANSAC in a subset; Section 3.3 reports 98.05%/98.17% for 5/7 manual p-points; Section 3.4 reports 98.95% for auto prompts; Section 3.6 states the overall 'average accuracy' is 96.9%; Section 4 reports 98.1%. It is unclear which metric is used for the headline claim, for which runs, and with what variation across fuels or droplet sizes. Please report the full per-run breakdown, the sample sizes for each percentage, and confidence intervals or standard deviations for the 96.9% headline value. Without these, the reader cannot assess whether the differences reflect genuine variation or artifacts of the evaluation protocol.","section":"Sections 3.1, 3.3, 3.4, 3.6, 4"},{"comment":"The definitions of IoU and Dice are incorrect as stated. The text defines IoU = |A∩B|/|A∪B| and Dice = 2|A∩B|/(|A|+|B|), where A is the segmentation mask and B is 'the manual-reference measurements value'. The manual-reference 'value' is a scalar flame diameter, not a binary mask, so set intersection and union are not defined. The IoU/Dice results in Figure 8 are therefore not interpretable as stated. Please either provide the actual manual segmentation masks used for these overlap metrics or define a proper mask-level ground truth. This is a load-bearing issue for the claim that auto-prompt segmentation has IoU>0.93.","section":"Section 3.4, Eq. (IoU/Dice definitions)"},{"comment":"The uncertainty budget conflates reference uncertainty with the method's own uncertainty. u_ref=0.085 is the uncertainty of the manual reference, not of the automated workflow; adding it to u_fit and u_cal yields a combined uncertainty that reflects agreement with a noisy reference rather than the repeatability or trueness of the automated method. Please separate the method's own uncertainty (e.g., repeatability from multiple runs, sensitivity to prompt placement, RANSAC fitting variability) from the validation comparison uncertainty. Also justify the assumed calibration standard deviations σL=0.5 mm and σP=1 pixel; these appear arbitrary but enter the final uncertainty directly.","section":"Section 3.7, Eq. (8)"}],"minor_comments":[{"comment":"The abstract says 'metrologically characterized' but the current uncertainty budget is dominated by the reference term; consider aligning the wording with the actual validation.","section":"Abstract/Introduction"},{"comment":"The parentheses in the indicator function are unbalanced: '1(ε_i(θ) ≤ τ)' should be '1(ε_i(θ) ≤ τ)' with closing parenthesis; also the sum index notation should be cleaned up.","section":"Section 3.1, Eq. (3)"},{"comment":"The text says 'B is the manual-reference measurements value'; please clarify whether a diameter value or a binary mask is used for the overlap metrics. If a mask is available, describe how it was obtained.","section":"Section 3.4"},{"comment":"The claim 'approximately 229-fold improvement' is based on manual measurement times from a previous study (Ref. [5]) and the authors' own workflow time. Please state clearly the hardware/software used for the timing comparison, since wall-clock time depends heavily on GPU vs CPU and on the manual measurement protocol.","section":"Section 2"},{"comment":"The increasing accuracy with initial droplet diameter is an interesting empirical observation, but the three explanatory factors are somewhat speculative. Consider quantifying the pixel-level boundary uncertainty as a function of droplet size instead of listing qualitative factors.","section":"Section 4, Figure 12d"},{"comment":"Some references are to arXiv preprints (e.g., Ref. [27]) and to a thesis (Ref. [29,34]) for the manual-reference uncertainty. If journal versions are available, cite them; otherwise, specify the exact source of u_ref=0.085 more precisely.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core idea is sound and the application domain is timely, but the validation logic is the main weakness. The authors appear to have selected hyperparameters using the same manual-reference data against which they report accuracy, and the uncertainty analysis includes the reference uncertainty as if it were the method's own. These issues are fixable in a revision, but they are load-bearing for the metrological claim. I would not consider acceptance without an independent validation or a substantial reframing of the claims."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThis is a genuinely useful engineering paper. The method — SAM2 with automatically generated boundary prompts, RANSAC circle fitting, and the video model for drift tracking — is new for droplet combustion imaging. The validation is substantial: 19,537 public ISS frames, three fuels, 45 runs, and the reported 96.9% mean relative agreement comes with an uncertainty budget and a 229x speedup. The authors also show that the auto-prompt version beats a manual-prompt version, and that the video model reduces drift-induced fluctuation. That is real progress for a tedious, operator-dependent measurement.\n\nThe soft spot is the ground truth. The m-measurements come from the same group's earlier work and carry an 8.5% uncertainty; the 3.1% MAPE is therefore agreement with a hand-drawn reference, not with an independent true diameter. The combined uncertainty of 8.54% is dominated by that reference term, so the automation itself could be far more precise than the headline suggests, or it could be tuned to the same group's interpretation. Without an independent reference, the 'metrologically characterized' claim is too strong. Also, the HSV thresholds, shrink factor, and RANSAC parameters are hand-set with no sensitivity analysis and no code release. The reader cannot test whether the result is stable to reasonable parameter changes. The Hough comparison is interesting, but the 55.9% baseline may be an unfair competition if the Hough parameters were not given the same tuning attention.\n\nThese issues are addressable, not fatal. The paper deserves a serious referee. I would ask for code, a sensitivity analysis of the free parameters, and either an independent validation set (synthetic images or a second group's manual measurements) or a rephrased claim that the agreement is relative to the m-measurements. With those changes it could become a standard tool in this niche.","headline":"Solid SAM2-based measurement pipeline for microgravity droplet flames, but the headline accuracy is only agreement with the same group's manual reference, not independent truth.","tokens_in":14219,"tokens_out":2184,"would_cite":true,"duration_ms":25164,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that an automatic prompt-point SAM2 workflow, with RANSAC circle fitting, can replace manual measurement of droplet flame diameter in microgravity combustion images, reaching 96.9% mean relative agreement to the manual refe","keywords":["droplet flame diameter","microgravity combustion","Segment Anything Model 2","automatic prompt generation","RANSAC circle fitting","image-based metrology","measurement uncertainty","combustion diagnostics"],"falsifier":"Resegment a subset of these videos with several independent human annotators and measure inter-annotator agreement; if annotators differ from each other by more than the reported 3.1% MAPE, part of the 'agreement' is an artifact of matching one particular reference. Alternatively, compare the SAM2 diameters against an independent, crisper-boundary measurement (e.g., UV or schlieren imaging of the same flames); if the independent reference deviates from both SAM2 and the manual reference by more than 8.5%, the claimed accuracy is not absolute.","tokens_in":13438,"feed_emoji":"🔥","tokens_out":5998,"duration_ms":58139,"temperature":0.7,"pith_summary":"The paper sets out to solve a measurement problem: flame diameter in microgravity droplet combustion is a key quantity, but its manual extraction from self-illuminated video is subjective, slow, and prone to bias from soot tails, blurred boundaries, and drifting droplets. The proposed answer is a fully automated pipeline built on Segment Anything Model 2, with automatically generated prompt points to locate the flame, a video memory mechanism to follow drifting droplets, and RANSAC circle fitting to discard soot-tail pixels. Across 19,537 images of n-heptane, n-decane, and n-octane, the workflow reports 96.9% mean relative agreement with manual-reference diameters, an 8.54% combined standard uncertainty, and roughly a 229-fold reduction in processing time. The authors present this as a metrologically characterized replacement for manual flame-diameter measurement and as a template for using AI segmentation in quantitative image-based metrology.","feed_headline":"AI workflow measures droplet flames 96.9% as accurately as humans","feed_subtitle":"A SAM2 pipeline with auto prompts and RANSAC replaces slow manual flame-diameter extraction in microgravity images.","key_machinery":"The load-bearing machinery is the automatic prompt-point generator coupled to SAM2's video memory. From a coarse HSV/brightness mask, the algorithm extracts the largest contour, estimates the flame centroid, places four symmetric boundary points (retracted by a fixed shrink factor), and adds a centroid point as positive prompts, plus two negative background points. These prompts remove the need for human point selection and suppress chamber-reflection artifacts, while SAM2's memory attention module keeps the segmentation temporally consistent as the droplet drifts. The final circle parameters come from RANSAC, which counts inliers with geometric residual below a pixel threshold and refits on","core_discovery":"The central claim is that a general-purpose segmentation model can be turned into a trustworthy digital measurement instrument for flame diameter without retraining. The authors combine an HSV-based contour estimate to generate five symmetric prompt points automatically, feed those prompts to SAM2's video model for frame-consistent segmentation of the flame, and then apply RANSAC-based circle fitting so that pixels belonging to a soot tail are rejected as geometric outliers rather than biasing the diameter. Against manual-reference measurements, the method achieves a mean relative agreement of 96.9% and a mean absolute percentage error of 3.1%, and it outperforms conventional Hough circle de","pith_inferences":["The 96.9% figure is agreement with a manual reference carrying 8.5% stated uncertainty; a fair reading is that the workflow reproduces human judgment at human-level uncertainty, not that it measures an absolute ground truth more accurately than a human can.","A natural extension, not tested here, is applying the same auto-prompt-plus-RANSAC recipe to other blurry-boundary circular measurements—bubbles in flow, weld pool shapes, or biological colonies—where a threshold-based contour can initialize prompts.","The dependence of accuracy on droplet size hints that pixel resolution, not segmentation quality, is the limiting factor for small droplets; higher-resolution imaging or super-resolution preprocessing would likely push small-droplet agreement toward the large-droplet level.","One could benchmark the auto-prompt strategy against a fully learned circle-regression baseline on the same 19,537 images; if a lightweight network matched or exceeded 96.9% at lower latency, the SAM2 dependency would be optional rather than necessary."],"forward_implications":["Flame diameter data from archived microgravity experiments can be regenerated automatically with documented agreement to earlier manual values, enabling comparisons across fuels and initial droplet diameters.","The workflow reduces a roughly 20,000-minute manual analysis load to about 90 minutes on the tested set, making full-video, high-throughput combustion diagnostics practical.","Accuracy rising with droplet size implies small-droplet flames are the hardest cases; for those, the method still reports around 96–97% agreement.","The reported combined uncertainty of 8.54% is dominated by the manual-reference uncertainty, so the automated pipeline itself adds little beyond the reference.","Because SAM2 is used without fine-tuning, the authors argue the same recipe can be applied to other fuels and other combustion-imaging tasks without per-dataset training."],"fun_headline_variants":["AI flame sizing matches humans 96.9% in microgravity","Automated SAM2 flame measurement hits 96.9% human accuracy","Flame diameter by AI: 96.9% agreement, 229x faster","SAM2+RANSAC measures droplet flames automatically","Self-prompting SAM2 flame metrology beats Hough"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the manual-reference diameters—carrying a stated 8.5% uncertainty—are the correct ground truth; the reported 96.9% agreement is agreement with hand-drawn measurements, not with an independently established physical flame diameter.","fun_headline_variants_meta":{"raw":{"variants":["AI flame sizing matches humans 96.9% in microgravity","Automated SAM2 flame measurement hits 96.9% human accuracy","Flame diameter by AI: 96.9% agreement, 229x faster","SAM2+RANSAC measures droplet flames automatically","Self-prompting SAM2 flame metrology beats Hough"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000764,"raw_usage":{"total_tokens":3262,"prompt_tokens":816,"completion_tokens":2446,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":560,"completion_tokens_details":{"reasoning_tokens":2355}},"tokens_in":560,"tokens_out":2446,"duration_ms":17195,"temperature":1.0,"reasoning_tokens":2355,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T20:30:13.396971+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Resegment a subset of these videos with several independent human annotators and measure inter-annotator agreement; if annotators differ from each other by more than the reported 3.1% MAPE, part of the 'agreement' is an artifact of matching one particular reference. Alternatively, compare the SAM2 diameters against an independent, crisper-boundary measurement (e.g., UV or schlieren imaging of the same flames); if the independent reference deviates from both SAM2 and the manual reference by more than 8.5%, the claimed accuracy is not absolute.","supporting_citations":[],"review_version":1}