{"id":"46f97515-e2ba-4897-a026-6bb86d155165","arxiv_id":"2504.18802","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Res-SAM uses SAM click prompts to propose candidate anomaly regions, then refines and classifies them with reservoir-computed wave-dynamics features, reporting detection accuracy above 85% on a 626-frame GPR dataset.","lead":"This paper combines a general-purpose image segmentation tool (SAM) with reservoir-computing features extracted from ground-penetrating radar waves to find and classify underground cavities, cracks, loose soil, pipes, and manhole covers. A smart generalist might read it because it promises accurate road-subsurface inspection with only a few human clicks and no heavy deep-learning training.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"A random frame-level split and an unreported threshold beta leave the >85% generalization claim unverified; a site-disjoint rerun with a fixed beta rule is needed.","rationale":"The paper has real merits: 2D-ESN is a lightweight unsupervised feature extractor, the SAM-prompt candidate step is sensible, the code link is provided, and the method avoids heavy training. I therefore read the claim in good faith. The reader's weakest assumption identifies feature-bank representativeness and the unreported beta; my concern agrees but sharpens it: the random frame-level split may exploit spatial autocorrelation in the B-scan acquisition, which is a known leakage mode in geophysical data. This is not an internal inconsistency in the algorithm, but it is a threat to the central generalization claim. A site-disjoint rerun is a single, decisive check. Since the reader already assigned CONDITIONAL, my stress-test reinforces that condition rather than changing the verdict; the paper should not be accepted outright without the split and beta being made explicit. I did not find a more fundamental flaw: the equations are coherent, and the method does not depend on circular reasoning beyond the normal evaluation protocol.","tokens_in":15223,"tokens_out":9694,"duration_ms":100727,"concrete_test":"Re-run the full Table 1 protocol with a site-disjoint split: assign whole survey lines or road segments to either the 20-frame initialization set or the test set, never both. Fix beta by a pre-registered rule on validation frames (for example, the 95th percentile of feature-bank nearest-neighbor distances) before computing any F1. If the 5/3 or 3/0 AUC/F1 drops by more than about 3 points or falls below the claimed 85% F1, the published numbers are inflated by frame-level leakage or threshold selection. At minimum, the authors should report the beta value used and the survey-line identities of the 20 initialization frames.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is empirical: Res-SAM maintains >85% detection accuracy and generalizes across environments using only 20 non-target frames. The load-bearing evaluation assumption is that the 20 frames randomly chosen in Section 2.3 are enough to characterize normal subsurface dynamics in all remaining frames, and that the Eq. (8) threshold beta is a fixed, well-defined constant. Both are insecure. The frames come from continuous road B-scans (Section 2.2, covering roughly 15 m each), so a random frame-level split can place initialization frames on the same road segment as test frames. Normal wave dynamics are spatially correlated along a survey line; the feature bank built from Eq. (4) can then contain near-duplicates of test-set normal patches. This artificially compresses the nearest-neighbor distances in Eq. (7) for normal patches, making the separation exploited by Eq. (8) look stronger than it would on a genuinely new site. The abstract's 'diverse environments' claim and the Section 3 deployment discussion therefore rest on a split that does not demonstrate cross-site transfer. Separately, beta is only called 'predefined' in Eq. (8); no value or selection procedure is reported. The reported F1=95.5% is threshold-dependent, so the headline >85% figure cannot be reproduced or checked for test-set threshold tuning from the text alone.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Res-SAM, a two-phase framework for detecting and categorizing subsurface anomalies in GPR B-scan data. In the first phase, a feature bank is built by fitting local patches from 20 non-target frames with a dual-directional echo state network (2D-ESN) and storing the fitted readout weights as normal dynamic features. In the second phase, SAM with click prompts proposes a candidate anomaly region; each point in that region is represented by a local patch fitted with 2D-ESN, and the resulting feature is scored by its L2 distance to the nearest feature in the bank, with a threshold beta deciding anomaly membership. Identified patches are merged into final rectangular regions, and a second 2D-ESN fitting is followed by clustering to categorize anomalies. Experiments on a 626-frame real-world dataset report AUC up to 0.896 and F1 up to 95.5% across prompt settings from 5/5 to 3/0, outperforming six interactive segmentation baselines, and clustering accuracy up to 0.91.","tokens_in":15530,"tokens_out":6025,"duration_ms":64822,"significance":"The central idea is plausible and practically motivated: Res-SAM does not require anomaly-labeled training data, uses only a small number of non-target frames for initialization, avoids training the reservoir, and combines SAM's visual prompting with wave-dynamics features. The paper also makes its code available, which is a concrete strength. If the empirical claims survive closer evaluation, the method would be a useful resource-efficient tool for GPR-based road inspection. The main weakness is not the method's internal logic but the evaluation protocol: the reported F1 depends on an unreported threshold, the data split may not be site-disjoint, and the comparison omits GPR-specific baselines from the cited literature. These issues are fixable within the scope of the manuscript, so the appropriate outcome is major revision rather than rejection.","major_comments":[{"comment":"The anomaly threshold beta is described only as 'predefined', and no value or selection procedure is reported anywhere in the manuscript. The headline F1 numbers in Table 1 (e.g., 95.5% in the 5/5 and 5/3 settings) are computed from the binary classifier in Eq. (8), so these claims cannot be reproduced or checked for test-set threshold tuning. Please report the beta value used for each prompt setting, state whether it was fixed before evaluation, and justify it a priori (for example, as a quantile of the feature-bank distance distribution). Reporting F1 across a range of beta or a precision-recall curve would also clarify how sensitive the result is to this parameter.","section":"Section 2.3/Table 1 and Section 4.3.3, Eq. (8)"},{"comment":"The generalization claim is not supported by the evaluation split. Section 2.2 states that frames cover approximately 15 meters of continuous road B-scan, and Section 2.3 states that 20 non-target frames were 'randomly chosen' from the dataset. If these initialization frames come from the same road segments as the test frames, the feature bank in Eq. (4) can contain near-duplicates of normal patches in the test set; the nearest-neighbor distances in Eq. (7) would then be artificially small for normal patches, inflating the separation exploited in Eq. (8). The paper's claim of applicability across 'diverse environments' requires a site-disjoint or survey-disjoint split, not a random frame-level split. Please also report results over repeated random initializations of the 20-frame bank, with means and standard deviations.","section":"Section 2.2 and Section 2.3"},{"comment":"The evaluation protocol is under-defined. The text says a detection is correct if its IoU with ground truth exceeds 0.5, but it does not specify how AUC and F1 are computed from this rule: are the units frames, candidate regions, or individual patches? How are frames with multiple anomalies handled, and what is the positive/negative definition for the ROC analysis? Without a precise scoring protocol, the numbers in Table 1 are not reproducible. Please specify the evaluation units and the exact procedure used to aggregate detections into AUC and F1.","section":"Section 2.3, Table 1"},{"comment":"The comparison is limited to interactive segmentation methods, which weakens the 'outperforms state-of-the-art' claim. The sentence 'there is a lack of research specifically focused on anomaly detection in GPR data' is contradicted by the authors' own references, including Refs. [18], [19], [25], and [26], which describe GPR anomaly detection methods. At least one or two recent GPR-specific detection baselines should be included under the same evaluation protocol, or the claim should be narrowed to 'outperforms interactive segmentation baselines'.","section":"Section 2.3"},{"comment":"The anomaly categorization experiments do not report the cluster count k used for K-Means, Agglomerative Clustering, and Fuzzy C-Means, nor do they describe how cluster labels were matched to ground-truth categories before computing accuracy, ARI, and NMI. Since k is a free parameter and these metrics depend on it, the clustering results in Table 2 cannot be reproduced as reported. Please state the value of k and the label-matching procedure.","section":"Section 2.4 and Section 4.3.4"},{"comment":"Several central hyperparameters are not reported: patch size X x Y, sliding stride s, reservoir dimensions and spectral radii of Wx and Wy, and the ridge regularization lambda in Eq. (3). These values are needed to reproduce the feature bank and the anomaly detection pipeline. Please provide them in the text or in a supplementary table.","section":"Section 4.1 and Section 4.2"}],"minor_comments":[{"comment":"The manuscript contains raw hexadecimal-looking path fragments after Fig. 3 and Fig. 6; these appear to be formatting corruption and should be removed.","section":"Figures 3 and 6"},{"comment":"The header for the FCM columns lists 'Acc NMI NMI'; this is likely a typo and should read 'Acc ARI NMI'.","section":"Table 2"},{"comment":"Several references are incomplete: Ref. [20] lacks publication details, Refs. [30] and [36] lack years or page ranges, and Ref. [36] lacks the conference proceedings. Please complete these entries.","section":"References"},{"comment":"The Discussion section's limitation paragraph notes that anomalies not indicated by human prompts are not automatically verified; this scope should be stated more prominently in the abstract or results, since the reported detection accuracy applies to prompted candidate regions rather than fully automatic scanning.","section":"Discussion"}],"recommendation":"major_revision","confidential_remarks":"The GitHub code availability is a plus, but the data availability statement only offers data 'on request', which makes the site-disjoint split issue harder to verify externally. If the authors cannot provide a site-disjoint evaluation or a fixed beta rule, the >85% generalization claim should be explicitly downgraded in the revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear Colleague,\n\nRes-SAM couples SAM's candidate region extraction with 2D-ESN dynamic features and a nearest-neighbor feature bank to detect and categorize subsurface anomalies in GPR B-scans. The core idea is reasonable: SAM gives a rough region from clicks, and the reservoir-computing features refine the boundary using local waveform dynamics rather than visual appearance alone. That integration is the genuinely new part; the 2D-ESN and model-space representation come from the authors' earlier work, and they cite it. The method is lightweight, requires no labeled anomalies, and appears robust to prompt reduction, which is useful for field practice.\n\nThe paper does a few things well. The method description is clear, the computations are straightforward, and the reported F1/AUC are strong across prompt settings. The categorization experiment is a nice extra, showing the same features cluster well by anomaly type. They also provide a code link, though I have not checked it.\n\nThe soft spots are in the evaluation. The threshold beta in Eq. (8) is called 'predefined' but its value or selection procedure is never reported. Since the headline F1 is threshold-dependent, the reader cannot tell whether the numbers come from a fixed rule or from test-set tuning. More importantly, the 20 non-target frames are chosen randomly from the same continuous B-scans used for testing. Subsurface conditions are spatially correlated along a survey line, so the feature bank may contain near-duplicates of normal patches that later appear in the test set. That artificially compresses the nearest-neighbor distances for normal patches and makes the anomaly separation look better than it would on a fresh site. The abstract's 'diverse environments' claim is not supported by a random frame-level split. There are also no repeated splits or error bars, and the baseline set omits the authors' own GPR-specific detectors from refs [18] and [25], which is a telling omission. The AUC/F1 protocol is under-specified: I cannot tell what unit is scored or how the negative set is formed, so Table 1 is not fully reproducible from the text alone.\n\nNone of this makes the method implausible, and I see no internal contradiction. It is an engineering integration that probably works in practice, but the empirical claims are stronger than the evidence. The paper deserves a serious referee, with the authors asked to report beta, use a site-disjoint split, add error bars, and compare with their prior GPR methods. If those are fixed, this becomes a solid applied contribution. For now, I would treat the >85% generalization claim with caution.\n\nBottom line: worth peer review, but only with major revision on the evaluation. I would not cite it until the beta and split issues are resolved.","headline":"Res-SAM is a sensible integration of SAM with 2D-ESN dynamic features, but the reported >85% generalization claim is not yet verified because beta is unreported and the split is not site-disjoint.","tokens_in":16064,"tokens_out":3327,"would_cite":false,"duration_ms":31872,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Res-SAM pairs SAM with a wave-dynamics reservoir to find GPR anomalies at F1 above 94 percent.","keywords":["Ground Penetrating Radar","GPR B-Scan data","Reservoir Computing","Subsurface Target Detection","Segment Anything Model","Anomaly detection","Echo State Network","Interactive segmentation"],"falsifier":"Take a second GPR survey collected on a different road surface or after a season change, use the original 20 frames as the feature bank with $\\beta$ unchanged, and measure F1 on the new data; if the score falls to the level of the SAM-only baseline or below the reported 94%, the normal-feature bank assumption fails. A cheaper check is to report the $\\beta$ value used and re-run the 3/0 experiment with $\\beta$ chosen by cross-validation on the 20 non-target frames.","tokens_in":15055,"feed_emoji":"📡","tokens_out":4604,"duration_ms":38646,"temperature":0.7,"pith_summary":"The paper tries to establish that underground anomalies in ground-penetrating radar (GPR) scans can be detected and categorized accurately using very little labeled data by combining a general-purpose visual segmenter with a signal-dynamics model. It argues that GPR images are not really images: they are recordings of electromagnetic waves, and the information that matters for anomalies lives in how the wave patterns change both along a single trace and across neighboring traces. Res-SAM first uses the Segment Anything Model with a few click prompts to get a rough candidate region, then fits every local patch with a dual-directional echo state network and compares the resulting dynamic features against a bank of features from normal frames. The paper reports that this two-stage approach reaches an AUC of 0.896 and an F1-score of 95.5% under moderate prompts, and remains at 0.866 AUC and 94.2% F1 when only three positive clicks are allowed. If true, this would make rapid, low-cost subsurface inspection practical in new environments with almost no training data.","feed_headline":"Res-SAM finds GPR underground hazards at F1 above 94 percent","feed_subtitle":"Adds a wave-dynamics reservoir to SAM, beating six interactive segmenters on real road scans.","key_machinery":"The load-bearing object is the Dual-Directional Echo State Network (2D-ESN), a reservoir-computing model whose hidden state at each point is computed from the current input and the hidden states of the horizontally and vertically preceding points, using two fixed random reservoirs. Fitting a patch by next-point prediction yields readout weights $[W_{out}\\,\\,a]$ that serve as a compact 'dynamic feature' of that patch. The argument runs on the assumption that normal subsurface conditions produce a tight cluster of such features, so the nearest-neighbor distance from a test patch's feature to the feature-bank determines whether that patch is anomalous; threshold $\\beta$ in Eq. (8) decides the cut and the merged patches define the final region.","core_discovery":"The central claim is that anomaly detection in GPR B-scan data is best treated as a comparison of local wave dynamics rather than a visual segmentation task, and that a reservoir-computing readout can supply the needed dynamic features. Res-SAM's pipeline is: (1) collect 20 non-target frames, extract patches with a sliding window, fit each with 2D-ESN, and store the readout weights in a feature bank; (2) let SAM propose a candidate region from click prompts; (3) for each point in that region, fit a centered patch with 2D-ESN and compute its anomaly likelihood as the L2 distance to the nearest feature in the bank; (4) merge patches whose likelihood exceeds a threshold beta; (5) refit the final region and cluster the resulting features to label anomaly type. The paper asserts that this consistently outperforms six interactive-segmentation baselines across all tested prompt settings, and that the category clustering reaches 0.91 accuracy, 0.85 ARI, and 0.89 NMI.","pith_inferences":["Beyond the paper: the nearest-neighbor threshold $\\beta$ is described as 'predefined' but its value and selection procedure are not reported; an honest comparison would require stating how $\\beta$ is set on the 20 normal frames and whether it is stable across sites.","Beyond the paper: the feature bank built from 20 frames is a small sample of normal variability; on roads with changing soil moisture, layering, or antenna coupling, the bank may need periodic refresh, and the paper does not test this drift.","Beyond the paper: the same two-stage idea (visual proposer plus reservoir dynamics refiner) could be applied to other non-visual imaging data such as ultrasonic or microwave tomography, where boundaries are also gradual.","Beyond the paper: a direct testable extension is to replace the hand-picked threshold $\\beta$ with a calibrated quantile of the bank distances, which would make the method parameter-free and easier to reproduce."],"forward_implications":["Deployment in a new urban area requires only a handful of normal GPR frames and a few clicks per frame, removing the labeled-anomaly bottleneck that limits deep-learning approaches.","Because the same 2D-ESN features are used for both detection and category clustering, the framework can label cavity, crack, looseness, pipeline, and manhole in a single pass without retraining.","The method's robustness to prompt reduction (F1 above 94% even at 3/0) implies that operators can rely on it in time-critical surveys where careful prompting is impossible.","The reported gains over SimpleClick and ScribblePrompt suggest that wave-dynamics features capture signal that pure visual encoders miss, which could transfer to other wave-based imaging modalities."],"supporting_citations":[{"why":"Supplies the visual candidate-region proposer that Res-SAM builds on.","marker":"[20]"},{"why":"Provides the reservoir-computing foundation that 2D-ESN extends.","marker":"[48]"},{"why":"Supplies the next-point prediction fitting and the idea of using readout weights as features.","marker":"[51]"},{"why":"Prior GPR model-space learning work that motivates treating GPR data as wave dynamics.","marker":"[25]"},{"why":"The strongest interactive-segmentation baseline that Res-SAM must beat.","marker":"[30]"},{"why":"Prior GPR anomaly detection context that defines the task difficulty and evaluation standard.","marker":"[8]"}],"fun_headline_variants":["Reservoir-enhanced SAM pinpoints GPR hazards at F1 94%","Res-SAM: wave-dynamics reservoir boosts GPR diagnosis","SAM meets reservoir computing for subsurface anomaly detection","High-accuracy GPR detection with minimal training data","Res-SAM beats SOTA on road scans using wave changes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole framework assumes that the 20 randomly chosen non-target frames capture all the normal wave dynamics that will appear in the remaining data, and that a single fixed distance threshold $\\beta$ cleanly separates normal variation from anomalies.","fun_headline_variants_meta":{"raw":{"variants":["Reservoir-enhanced SAM pinpoints GPR hazards at F1 94%","Res-SAM: wave-dynamics reservoir boosts GPR diagnosis","SAM meets reservoir computing for subsurface anomaly detection","High-accuracy GPR detection with minimal training data","Res-SAM beats SOTA on road scans using wave changes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001344,"raw_usage":{"total_tokens":5478,"prompt_tokens":982,"completion_tokens":4496,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":598,"completion_tokens_details":{"reasoning_tokens":4412}},"tokens_in":598,"tokens_out":4496,"duration_ms":31243,"temperature":1.0,"reasoning_tokens":4412,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:09:22.067605+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a second GPR survey collected on a different road surface or after a season change, use the original 20 frames as the feature bank with $\\beta$ unchanged, and measure F1 on the new data; if the score falls to the level of the SAM-only baseline or below the reported 94%, the normal-feature bank assumption fails. A cheaper check is to report the $\\beta$ value used and re-run the 3/0 experiment with $\\beta$ chosen by cross-validation on the 20 non-target frames.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the visual candidate-region proposer that Res-SAM builds on."},{"cited_title":"echo state","cited_arxiv_id":null,"evidence_quote":"Provides the reservoir-computing foundation that 2D-ESN extends."},{"cited_title":"& Yao, X","cited_arxiv_id":null,"evidence_quote":"Supplies the next-point prediction fitting and the idea of using readout weights as features."},{"cited_title":"& Chen, H","cited_arxiv_id":null,"evidence_quote":"Prior GPR model-space learning work that motivates treating GPR data as wave dynamics."},{"cited_title":"& Niethammer, M","cited_arxiv_id":null,"evidence_quote":"The strongest interactive-segmentation baseline that Res-SAM must beat."},{"cited_title":"& Chen, H","cited_arxiv_id":null,"evidence_quote":"Prior GPR anomaly detection context that defines the task difficulty and evaluation standard."}],"review_version":1}