{"id":"8f10e40b-4817-4056-8832-7aeb5f21a731","arxiv_id":"2608.04216","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"An automated SEM analysis pipeline using PerSplat and Sauvola thresholding is presented, but its validation is circular and the reported porosity at 1400°C is nine times the manual value.","lead":"The authors built an automated image-analysis pipeline to measure porosity, grain size, and pore size from SEM images of sintered alumina, and they report it as matching manual expert segmentation. The validation is undercut because the pipeline's parameters were tuned on the same images used for scoring, so the headline agreement is not an independent result.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported 1400°C IoU (99.35% mean) exceeds the maximum possible IoU implied by the paper's own porosity values (max 97.8%); the validation claim is internally inconsistent.","rationale":"The reader's weakest_assumption (hyperparameter tuning in §2.4) identifies a real generalization gap, since k and w were tuned on the same manual segmentations that produce Table 2. That alone would justify rejecting the claim of validated agreement. However, the more load-bearing problem is that the reported numbers contradict each other arithmetically. The 1400°C mean IoU of 99.35% cannot be achieved by any two binary masks whose pore fractions differ by an order of magnitude (0.27% vs 2.44%). The bound holds for either choice of positive class: pores give max IoU ≈ 0.11, solid gives max ≈ 0.978. This is not a matter of statistical generalization or distribution shift; it indicates an error in the reported measurements, metric, or table transcription. Because the central claim is that the workflow agrees with manual segmentation, and the evidence for that agreement is internally impossible, the reader's REJECT verdict is correct and should remain unchanged. A reanalysis with held-out data and pre-registered parameters would address the tuning issue, but the arithmetic inconsistency must be resolved first.","tokens_in":9546,"tokens_out":10652,"duration_ms":84601,"concrete_test":"Obtain the binary masks generated by the pipeline and the manual masks used for Table 2, then recompute per-image IoU and the per-image pore-area fractions. If the 1400°C predicted mask covers ~2.4% of pixels and the manual mask covers ~0.27%, the mean IoU must be ≤ ~0.11 (or ≤ ~0.978 if the solid class is scored); a value of 99.35% would indicate that Table 2 used different masks, a different class label, or a different metric than Table 3. No fresh data are needed: this is a direct arithmetic re-derivation from the paper's own reported values.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Table 2 reports mean IoU 99.35% (min 98.64%) for the 1400°C images, while Table 1 reports manual porosity 0.27±0.19% and Table 3 reports the workflow porosity as 2.44±0.68% for the same condition. For any binary segmentation, the Jaccard index between predicted and ground-truth masks cannot exceed min(A_pred,A_true)/max(A_pred,A_true), where A is the area fraction of the positive class. If the positive class is pores, this bound is 0.0027/0.0244 ≈ 0.11; if the positive class is solid (the other label), the bound is (1−0.0244)/(1−0.0027) ≈ 0.978. Neither bound reaches 0.9935, so the reported IoU values cannot be correct for the same masks that produced the reported porosity values, even before considering the hyperparameter tuning admitted in §2.4. The abstract's cited IoU values are also the maxima from Table 2, not the means. The central claim of agreement with expert manual segmentation therefore rests on internally inconsistent validation numbers.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents an automated image-analysis workflow for extracting porosity, grain size, and pore size distributions from SEM images of sintered alumina. The workflow combines noise filtering (selecting PerSplat), Sauvola local thresholding, morphological pore reconstruction, and intercept-based grain size measurement. The authors validate their pipeline against manual ImageJ segmentations for samples sintered at 1200°C and 1400°C, reporting Intersection over Union (IoU) scores of 95.14% and 99.85% in the Abstract, and conclude that the workflow is in agreement with expert manual segmentation. Additional comparisons are made with NIST GSAT for grain size and PoreSpy for pore properties. The central claim is that the automated pipeline can replace manual microstructure characterization for generating materials datasets.","tokens_in":9819,"tokens_out":6625,"duration_ms":52586,"significance":"If the reported agreement with manual segmentation were valid, this workflow would be a useful contribution to automated microstructure characterization, an area of active interest for materials informatics. The paper draws on established open-source tools, provides a documented five-phase pipeline, and attempts quantitative comparisons against three reference methods (manual ImageJ analysis, NIST GSAT, and PoreSpy). The strengths are the reproducible use of standard image-processing libraries, the explicit description of hyperparameters, and the clear presentation of the application context. However, the validation is undermined by a mathematical inconsistency between the reported IoU values and the reported porosity values, and by a circular evaluation protocol in which the same manual segmentations are used both to tune the parameters and to score the results. These issues directly affect the central claim and prevent the paper, in its current form, from substantiating its stated contribution.","major_comments":[{"comment":"The reported IoU values for 1400°C (mean 99.35%, min 98.64%) are mathematically incompatible with the reported porosity values. With the workflow porosity of 2.44±0.68% (Table 3) and the manual porosity of 0.27±0.19% (Table 1), the maximum possible IoU is min(A_pred,A_true)/max(A_pred,A_true): about 11% if the positive class is pores, and at most (1−0.0244)/(1−0.0027) ≈ 97.8% if the positive class is solid. Neither bound reaches 98.64%, so the IoU values in Table 2 cannot have been computed from the same segmentation masks that produced the porosity statistics in Tables 1 and 3. This internal inconsistency invalidates the paper's central validation claim.","section":"Section 3.2, Tables 1 and 3"},{"comment":"The validation is circular. Section 2.4 states that the Sauvola parameters k and w were tuned 'for each temperature class to compare against manually thresholded images,' and Section 3.1 states that PerSplat was selected because it 'yielded the porosity value closest to the manual segmentation.' Table 2 then reports IoU scores against those same manual segmentations, with no held-out images or cross-validation. The IoU values therefore reflect fit to the labels used for optimization, not independent agreement with expert segmentation. This does not establish the claimed generalizable agreement in the Abstract and Conclusion.","section":"Sections 2.4 and 3.1"},{"comment":"The Abstract reports IoU scores of 95.14% and 99.85%, which are the maximum values in Table 2, not the mean values (92.18% and 99.35%). Reporting the best-case values as the overall validation result overstates the performance and is inconsistent with the summary statistics the authors themselves present.","section":"Abstract and Table 2"},{"comment":"The porosity values for PerSplat in Table 1 (0.03±0.01 for 1200°C and 0.0044±0.0029 for 1400°C, presumably as fractions) are inconsistent with the workflow porosity in Table 3 (12.4±2.7% and 2.44±0.68%). The paper does not explain whether Table 1 reports porosity after filtering alone, after thresholding alone, or with different units; if the values are fractions, 0.03 would be 3%, not 12.4%. This inconsistency makes it impossible to interpret the filter comparison and further undermines the validation narrative.","section":"Tables 1 and 3"}],"minor_comments":[{"comment":"There is a typo: 'minium gap filter' should be 'minimum gap filter.'","section":"Section 3.3"},{"comment":"The Conclusion refers to 'validating the synthetic data generation,' but this paper does not describe any synthetic data generation; this phrase appears to be a leftover from a different context and should be corrected.","section":"Conclusion"},{"comment":"The grain-size analysis uses several per-class free parameters (binary map thresholds 20 and 35, minimum gap filters of 3 and 12 pixels) whose selection process is not described; please clarify whether these were also tuned against the manual segmentations, since this would extend the circularity concern.","section":"Sections 2.5 and 3.3"},{"comment":"The table would be clearer if it stated explicitly whether porosity values are fractions or percentages, since the manual row mixes a fraction (0.09) with the corresponding percentage (9%) in later tables.","section":"Table 1"},{"comment":"The text cites 'Choi and Choi' in the context of watershed segmentation, but reference [20] in the bibliography is Heilbronner, R., 'Automatic grain boundary detection and grain size analysis...'; the citation appears mismatched.","section":"Introduction, reference [20]"}],"recommendation":"reject","confidential_remarks":"The internal inconsistency between the IoU values and the porosity data is decisive: the reported IoU numbers cannot be correct for the same masks that produced the porosity statistics. Even a thorough revision would require a new validation protocol, retraining or re-tuning on a held-out set, and recomputation of the headline numbers, which is beyond a normal revision. I recommend rejection, although the overall topic is within the journal's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Hi,\n\nOne thing you should know: the paper's central validation claim is contradicted by its own tables. At 1400°C, Table 2 reports mean IoU 99.35% (min 98.64%) against manual segmentation, while Table 1 lists manual porosity as 0.27% and Table 3 gives the workflow's porosity as 2.44%. For binary masks, the Jaccard index cannot exceed min(A_pred,A_true)/max(A_pred,A_true). With pores as the positive class that bound is about 0.11; with solid as positive it is still only about 0.978. Either way, the reported IoU of 0.9935 is mathematically impossible for the same masks that produce those porosity numbers. The abstract quotes the maximum IoU (99.85%) rather than the mean. This is not a nitpick; it sinks the paper's central claim.\n\nThere is substance worth acknowledging. The pipeline sensibly combines PerSplat topological filtering, Sauvola local thresholding, morphological reconstruction for pores, and the intercept method for grain size, and it runs this on 375 real SEM images of sintered alumina. For the materials-informatics community, that is the kind of legwork that could help fill the labeled-data bottleneck. The methods are described clearly enough to reproduce in principle.\n\nThe soft spots are serious, though. The Sauvola parameters k and w were tuned per temperature class against the same manual segmentations used to compute the IoU scores, and the filter choice (PerSplat) was made because it gave porosity closest to manual. So the numbers are fits, not generalization estimates. No code or data is provided. Beyond the 1400°C paradox, the 1200°C pore diameter is roughly half the manual value, and the final 1400°C porosity is nine times the manual value. The authors honestly report these discrepancies but never confront the tension with the IoU claim. The citation pattern is fine, and the underlying methods are standard.\n\nWho gets value here? Someone looking for concrete parameter choices for SEM segmentation might skim the pipeline, but they should ignore the validation. The REJECT verdict is right. If the authors redo the evaluation with held-out images, fix the IoU/porosity inconsistency, and release code and ground truth, there is a usable tool in here. As written, I would desk-reject it; the internal contradiction is exactly what a referee should not have to spend a report on. I'd tell the authors to fix the validation and resubmit.","headline":"The pipeline is a reasonable engineering effort, but its validation is impossible: the reported IoU at 1400°C cannot coexist with the reported porosity values.","tokens_in":10350,"tokens_out":7393,"would_cite":false,"duration_ms":56262,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An automated pipeline segments sintered ceramic SEM images and extracts microstructure statistics in agreement with expert manual analysis.","keywords":["image analysis","microstructure characterization","additive manufacturing","scanning electron microscopy","thresholding","porosity","grain size","sintered ceramics"],"falsifier":"Hold out a subset of SEM images per temperature class, tune the Sauvola parameters only on the remaining images, and compute IoU on the held-out set against manual segmentations; if the held-out IoU drops substantially below the reported 92.18% and 99.35% means, the claimed agreement with expert analysis is an artifact of parameter tuning rather than a general property of the workflow.","tokens_in":9339,"feed_emoji":"🔬","tokens_out":9521,"duration_ms":72871,"temperature":0.7,"pith_summary":"The paper aims to automate the slow, subjective work of measuring porosity, solid fraction, grain size, and pore size from scanning electron microscopy images of sintered ceramics. The obstacle is that grain and pore phases overlap in grayscale intensity, producing unimodal histograms that global thresholding cannot separate. The proposed workflow combines PerSplat topological filtering with Sauvola local adaptive thresholding and morphological reconstruction, then validates the results against manual ImageJ segmentations. The authors report Intersection over Union scores of 95.14% and 99.85% in the abstract, with mean values of 92.18% and 99.35% in the results table, and physical property estimates that track the manual reference data. If the agreement holds, the workflow gives materials science a high-throughput path to the annotated microstructure datasets needed for machine learning.","feed_headline":"Automated pipeline matches manual ceramic microstructure analysis","feed_subtitle":"PerSplat plus Sauvola thresholding extracts porosity and grain size from SEM images, matching expert manual results.","key_machinery":"The load-bearing combination is PerSplat topological filtering plus Sauvola local adaptive thresholding, followed by morphological reconstruction for pore identification. PerSplat uses persistent homology to discard noise while keeping topologically significant features; Sauvola sets a threshold from the local mean and standard deviation in a window around each pixel, which is what makes binarization work for unimodal histograms. Pore detection uses dual thresholding with a Sauvola boundary mask and a percentile seed mask, then iterative geodesic dilation to reconstruct true pore shapes. Grain size is measured with a random line intercept method and a minimum gap filter, and the pipeline is validated against ImageJ manual segmentation and compared with PoreSpy and GSAT tools.","core_discovery":"The central claim is that a combination of topological noise filtering and local adaptive thresholding is sufficient to segment microstructures whose phases cannot be separated by global intensity thresholds. Sauvola thresholding computes a per-pixel threshold from the local mean and standard deviation, which handles overlapping intensity ranges, while PerSplat removes noise by topological persistence and preserves significant features. The paper reports IoU values against manual segmentations of 92.18% mean at 1200°C and 99.35% mean at 1400°C (with maxima 95.14% and 99.85% cited in the abstract), and shows that the pipeline's porosity, grain size, and pore size measurements are closer to manual reference values than the PoreSpy and GSAT comparison tools. The conclusion states that the workflow is in agreement with expert manual segmentation results.","pith_inferences":["The reported IoU values depend on per-temperature-class tuning of the Sauvola parameters k and w, so a reader should test the pipeline on held-out images before treating the agreement with manual analysis as a general capability.","Validation on only two sintering temperatures of one material means the workflow's transferability to other ceramics, magnifications, detectors, or charging conditions remains an open question.","The gap between the mean and maximum IoU in the results suggests some images are substantially harder; reporting per-image failure modes could help target future improvements.","The grain size portion of the workflow still relies on several hand-set thresholds and gap filters, so the pipeline automates segmentation but not every step of characterization."],"forward_implications":["Materials labs can generate annotated microstructure datasets at high throughput, removing a major bottleneck in training machine learning models for processing-structure-property relationships.","The same local-thresholding-plus-topological-filtering approach could be applied to other material systems whose phases overlap in image intensity, not just sintered alumina.","The per-pore and per-grain measurements the pipeline produces enable statistical comparison of microstructures across processing conditions such as sintering temperature.","The workflow's speed makes it practical to analyze large image stacks, supporting future three-dimensional characterization from tomographic data.","The authors state a next step of conditional generative models for microstructures, which would rely on datasets this pipeline can produce."],"supporting_citations":[{"why":"Supplies the Sauvola local adaptive thresholding method that binarizes images with overlapping phase intensities.","marker":"[34]"},{"why":"Supplies PerSplat, the topological filtering method used to remove noise while preserving microstructural features.","marker":"[30]"},{"why":"ImageJ provides the manual segmentation reference values against which the automated pipeline is scored.","marker":"[36]"},{"why":"PoreSpy is the comparison toolkit used to benchmark the pore space measurements of the workflow.","marker":"[17]"},{"why":"The GSAT grain size analysis tool is the comparison baseline for the workflow's grain size measurements.","marker":"[26]"},{"why":"Morphological grayscale reconstruction underpins the iterative geodesic dilation used to reconstruct true pore regions.","marker":"[40]"},{"why":"Provides the dual-threshold morphological reconstruction approach used in pore detection.","marker":"[14]"}],"fun_headline_variants":["Automated ceramic imaging matches manual analysis","Pipeline segments ceramic microstructures automatically","AI workflow matches manual ceramic porosity reads","Topological filter + Sauvola segments ceramics","Fast ceramic microstructure extraction from SEM"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The evaluation assumes that the manual segmentations used to score the pipeline are independent of the manual segmentations used to tune the Sauvola parameters, but the paper states the parameters were tuned per temperature class to compare against manually thresholded images, so the IoU scores may reflect fitting rather than generalization.","fun_headline_variants_meta":{"raw":{"variants":["Automated ceramic imaging matches manual analysis","Pipeline segments ceramic microstructures automatically","AI workflow matches manual ceramic porosity reads","Topological filter + Sauvola segments ceramics","Fast ceramic microstructure extraction from SEM"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000344,"raw_usage":{"total_tokens":1868,"prompt_tokens":903,"completion_tokens":965,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":519,"completion_tokens_details":{"reasoning_tokens":904}},"tokens_in":519,"tokens_out":965,"duration_ms":6854,"temperature":1.0,"reasoning_tokens":904,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T00:13:41.517332+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold out a subset of SEM images per temperature class, tune the Sauvola parameters only on the remaining images, and compute IoU on the held-out set against manual segmentations; if the held-out IoU drops substantially below the reported 92.18% and 99.35% means, the claimed agreement with expert analysis is an artifact of parameter tuning rather than a general property of the workflow.","supporting_citations":[{"cited_title":"Topological Filtering for 3D Microstructure Segmentation","cited_arxiv_id":"2104.13430","evidence_quote":"Supplies PerSplat, the topological filtering method used to remove noise while preserving microstructural features."},{"cited_title":"Protein Science : A Publication of the Protein Society30(1), 234–249 (Jan 2021)","cited_arxiv_id":null,"evidence_quote":"ImageJ provides the manual segmentation reference values against which the automated pipeline is scored."},{"cited_title":"Journal of Open Source Software4, 1296 (May 2019)","cited_arxiv_id":null,"evidence_quote":"PoreSpy is the comparison toolkit used to benchmark the pore space measurements of the workflow."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The GSAT grain size analysis tool is the comparison baseline for the workflow's grain size measurements."},{"cited_title":"IEEE Transactions on Image Processing2(2), 176–201 (Apr 1993)","cited_arxiv_id":null,"evidence_quote":"Morphological grayscale reconstruction underpins the iterative geodesic dilation used to reconstruct true pore regions."},{"cited_title":"Image Analysis and Stereology27(2), 79– 85 (2008)","cited_arxiv_id":null,"evidence_quote":"Provides the dual-threshold morphological reconstruction approach used in pore detection."}],"review_version":1}