{"id":"baaf92c9-634b-4c11-9a51-ced7b5d77708","arxiv_id":"2506.14289","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"On two investigative drilling boreholes, isolation forest with automatic settings detects labeled anomalies better than 3-sigma and IQR rules, and adding two-cluster K-means removes soil drilling data.","lead":"This paper tests three machine-learning anomaly detectors, isolation forest, one-class SVM and DBSCAN, against traditional statistical rules for cleaning investigative drilling data from two boreholes. It finds that isolation forest in automatic mode performs best and that adding a two-cluster K-means step removes soil drilling sections without extra tuning.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"IsoForest's 'best-performing' claim hinges on 1–2 labeled points per borehole; manual labels and lack of held-out validation make the ranking statistically unsupported.","rationale":"The reader's conditional verdict is appropriate. The ML-versus-statistical comparison is likely robust, since the gaps are large (e.g., 3-sigma detects 13/28 anomalies and IQR mislabels 34/162 normal points versus 23-26/28 and 2-6/162 for the ML methods). The fragile part is the further claim that IsoForest is 'best-performing' and 'proven' so: the differences separating IsoForest from DBSCAN are only one to three points, which is exactly the scale of uncertainty in manual depth-profile labels. The proposed one-sample label-shift test would directly establish whether the ranking survives that uncertainty, and the caveat in the concrete test acknowledges that generalization to new boreholes still requires additional data. Therefore, the reader's CONDITIONAL verdict correctly captures the gap between the evidence and the abstract's stronger wording, and no change to the verdict is needed.","tokens_in":12891,"tokens_out":9311,"duration_ms":105412,"concrete_test":"Perform a label-sensitivity analysis on the two existing boreholes: for each anomaly window boundary in Table 2, shift the boundary by one depth sample (flipping one labeled point), recompute recall and FPR for the five methods, and record whether the IsoForest-vs-DBSCAN ranking changes. Then add exact binomial 95% confidence intervals for FPR. If a single one-sample shift changes the best method, the 'best-performing' conclusion is not robust; if it never does, the label-noise concern is answered for these two boreholes, but a held-out borehole is still needed to support generalization.","verdict_should_be":"UNCHANGED","load_bearing_attack":"All metrics in Tables 3 and 4 are computed against a manually labeled partition of two boreholes (Section 2, Table 2). The IsoForest-over-DBSCAN ranking is numerically precarious: in Borehole A, IsoForest detects 23/28 anomalies with 4/162 false positives, while DBSCAN detects 26/28 with 6/162; in Borehole B, IsoForest detects 22/26 with 2/118, while DBSCAN detects 21/26 with 2/118. Thus the entire advantage is a difference of two false positives in one borehole and one true positive in the other. Since labels were assigned by visual inspection of depth profiles, a one-sample shift at a collaring, rod-adding, or transition boundary can change exactly those counts. Additionally, one-class SVM and DBSCAN hyperparameters were selected via trial-and-error on these same boreholes to match IsoForest's rod-adding behavior (Sections 4.1 and 4.2), so the comparison does not reflect an automatic, out-of-the-box workflow and is not a fair estimate of out-of-borehole performance. With n=2 boreholes and no confidence intervals, the abstract's 'proven to be the best-performing algorithm' is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper investigates automatic cleaning of investigative drilling (ID) data using three unsupervised anomaly-detection algorithms (IsoForest, one-class SVM, DBSCAN) and compares them with the 3σ rule and IQR method on two manually labeled boreholes. For rock-only borehole A, the task is anomaly removal; for mixed soil/rock borehole B, the paper proposes a hybrid IsoForest + two-cluster K-means strategy to remove both anomalies and soil drilling data. The paper reports confusion-matrix counts and runtimes, concluding that all three machine learning methods outperform the statistical baselines, that IsoForest performs best in auto mode without hyperparameter tuning, and that the hybrid strategy enables fully automatic cleaning of rock-dominated ID datasets.","tokens_in":13112,"tokens_out":5194,"duration_ms":54838,"significance":"If the claims held, the paper would offer a practical, automatic cleaning pipeline for rock-dominated investigative drilling data, supporting construction of large ML-ready geotechnical datasets. The study has clear strengths: it carefully enumerates four types of drilling-action anomalies; it uses standard, reproducible scikit-learn implementations; it reports detailed confusion counts per anomaly source; and it tests the hybrid method on a second borehole to check over-cleaning. The main limitation is the evidence base: all quantitative conclusions rely on only two manually labeled boreholes, with no held-out data, no cross-validation, and no uncertainty quantification. The contribution is therefore best viewed as a promising proof-of-concept rather than a definitive comparative evaluation.","major_comments":[{"comment":"The central claim that IsoForest is the best-performing algorithm and that all ML methods outperform statistical methods rests on only two boreholes with manually assigned labels. The reported metrics are computed on the same data used for visual labeling and hyperparameter selection. The numerical ranking among algorithms is fragile: in borehole A, DBSCAN detects 26/28 anomalies versus IsoForest's 23/28, with 6 versus 4 false positives; in borehole B, DBSCAN detects 21/26 versus IsoForest's 22/26 with identical false positives. With n=2 and no confidence intervals or resampling, the abstract's statement that IsoForest is 'proven to be the best-performing algorithm' is not supported. The authors should either validate on additional boreholes with label reliability assessment or substantially temper the claims in the abstract and conclusions.","section":"Section 4.1, Tables 3–4; Section 2, Table 2"},{"comment":"One-class SVM (nu) and DBSCAN (eps, min_samples) were tuned by trial and error on the same boreholes to match IsoForest's rod-adding baseline, and their final scores are reported on those same boreholes. This makes the comparison favorable to IsoForest by construction: the competing methods are hand-fit to the test data while IsoForest runs in auto mode. The statement in the abstract that one-class SVM and DBSCAN 'required hyperparameter tuning' is accurate, but the subsequent claim that IsoForest is superior for automatic cleaning is not a fair out-of-the-box comparison. A proper evaluation would use a separate validation split for tuning or default hyperparameters for all algorithms, and would report performance on held-out boreholes.","section":"Section 4.1, one-class SVM and DBSCAN tuning; Section 4.2"},{"comment":"The anomaly labels were created by visual inspection of depth profiles without an independent physical measurement or inter-rater reliability check. Since every recall and false-positive count in Tables 3 and 4 is defined relative to these labels, the evaluation inherits the subjectivity of the labeling. This is consequential because some labeled anomalies (e.g., collaring onset, rod-adding) are contiguous with normal transitions; a one-point shift in labeling could alter the ranking between IsoForest and DBSCAN in either borehole. The authors should describe the labeling protocol in more detail, ideally with independent labels from a second expert or with quantitative criteria based on the known operational actions.","section":"Section 2, Table 2"},{"comment":"The paper does not provide a single aggregate performance metric (e.g., F1-score, Matthews correlation coefficient, or a cost-weighted score) or a significance test to support the statement that 'all three machine learning algorithms outperformed traditional statistical methods' in both tasks. For example, in borehole A the IQR method achieves 100% recall but misclassifies 34 of 162 normal points, while IsoForest has 82% recall and only 4 false positives; whether this constitutes 'outperforming' depends on the relative cost of false positives and false negatives. Without an explicit criterion or metric, the superiority claim is a qualitative judgment rather than a demonstrated result.","section":"Tables 3–4 and Figure 10"}],"minor_comments":[{"comment":"The phrase 'tuned to the baseline performance of IsoForest detailed forehead' contains a typo; 'forehead' should be 'aforementioned' or 'above'.","section":"Section 4.1, DBSCAN paragraph"},{"comment":"Equation (3) is garbled in the text; it should read approximately c(n) = 2H(n−1) − 2(n−1)/n. Please correct the typesetting.","section":"Section 3.2.1, Eq. (3)"},{"comment":"The 'auto mode' of IsoForest is not explicitly defined. It would be helpful to state which scikit-learn parameters (e.g., contamination, n_estimators, max_samples) are left at defaults, so readers can reproduce the 'no hyperparameter adjustment' claim.","section":"Abstract and Section 4.1"},{"comment":"For the 'IsoForest combined with two-cluster K-means' row, the hyperparameter column says 'None', but K-means uses k=2. List the full parameter set for reproducibility.","section":"Table 4, last row"},{"comment":"The normalized confusion matrices should specify whether rows and columns correspond to actual or predicted labels, and the text's reference to the 'bottom-right quadrant' for true positive rate should be checked against the figure's orientation.","section":"Figure 10"},{"comment":"The spelling of 'one-class SVM' is inconsistent ('one-class SVM', 'one-Class SVM'); please unify. Also, the abstract phrase 'hyperparameter tuning s were required' has an agreement error.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a practical engineering application with a clear domain motivation, but the comparative claims are substantially ahead of the evidence. The paper would be strengthened by additional boreholes or by reframing as a proof-of-concept without claiming a proven ranking. If the authors can add even two or three more labeled boreholes with a simple cross-validation or a stated aggregate metric, the central claims would become defensible."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this is the first application I know of standard anomaly-detection algorithms to cleaning investigative drilling (MWD) data, and the proposed IsoForest plus two-cluster K-means hybrid is a practical idea worth writing up. But the paper's headline claim that IsoForest is 'proven to be the best-performing algorithm' is not supported by the evidence. Only two boreholes, manual labels, no held-out data, no cross-validation, and the performance differences between IsoForest and DBSCAN are a false positive here and a true positive there. The conclusion overreaches.\n\nWhat the paper does well: the problem is real, and the authors explain it clearly. They identify the four operational actions that create anomalies in the data stream, use standard algorithms in a sensible way, and report confusion-matrix metrics. I also give them credit for noting that the optimal hyperparameters for one-class SVM and DBSCAN vary between boreholes, which is the real practical argument for IsoForest in an automatic cleaning pipeline. The hybrid strategy for removing soil is a reasonable engineering solution, and testing it on a rock-only borehole to check for over-cleaning is a good instinct.\n\nThe soft spots are the evaluation design. The labels come from visual inspection of depth profiles, so all the recall and false-positive figures inherit whatever subjectivity is in those labels. One-class SVM and DBSCAN were tuned on the same boreholes they are evaluated on, which does not reflect an automatic workflow and makes the comparison hard to interpret. And the numerical difference between IsoForest and DBSCAN is precarious: in Borehole A, DBSCAN finds 26/28 anomalies with 6 false positives versus IsoForest's 23/28 with 4; in Borehole B, IsoForest finds 22/26 with 2 false positives versus DBSCAN's 21/26 with 2. That is a wash, not a 'proven best'. With n=2, there are no error bars and no way to know whether the ranking generalizes. No code or data are provided, which makes reproduction impossible.\n\nThis paper is for geotechnical practitioners and applied-ML people working on MWD processing. They will find the anomaly taxonomy and the hybrid workflow useful. A serious referee should engage, because the problem matters and the approach is transferable, but the revision needs more boreholes, independent hyperparameter selection, and a claim that matches the evidence. I would send it to peer review with a clear request for major revision.\n\nRecommendation: engage with it, but make the authors earn the conclusion.","headline":"A legitimate first application of off-the-shelf anomaly detectors to investigative drilling data, but the 'IsoForest is best' claim rests on two manually labeled boreholes and margins of one or two points; the paper deserves peer review but needs a much more careful evaluation and a softer conclusion.","tokens_in":13715,"tokens_out":3174,"would_cite":false,"duration_ms":32364,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Machine learning beats statistical rules at cleaning investigative-drilling data.","keywords":["investigative drilling","measurement while drilling","data cleaning","anomaly detection","isolation forest","one-class SVM","DBSCAN","rock drilling"],"falsifier":"Run the same five cleaning methods on a third borehole whose anomalies are identified by an independent source such as core recovery logs, a drilling diary, or a down-hole camera, and compare recall and false positives; if auto-mode IsoForest's recall drops well below the 23/28 figure reported here, or if a per-borehole tuned SVM or DBSCAN consistently beats it, the paper's central ranking is not stable.","tokens_in":12669,"feed_emoji":"⛏️","tokens_out":6799,"duration_ms":62743,"temperature":0.7,"pith_summary":"Investigative drilling produces a near-continuous depth record of penetration rate and percussion, feed, and rotation pressures, but non-informative stretches—drill collaring, unexpected pauses, rod adding, end-of-drilling, and any soil section in a rock hole—contaminate the stream. The paper asks whether unsupervised machine learning can remove those stretches automatically, and claims the answer is yes: on two example boreholes, isolation forest, one-class SVM, and DBSCAN all beat the standard 3-sigma and interquartile-range rules at detecting anomalies while keeping normal data. Isolation forest is singled out as the best of the three because it reaches 23 of 28 anomalies and only 4 false positives on the rock-only borehole with default settings, whereas the other two algorithms need per-borehole hyperparameter tuning. For mixed soil–rock holes, the paper adds a two-cluster K-means step after isolation forest, which separates and removes all soil drilling data without additional false positives. The broader point is practical: if these results hold across many boreholes, manual data cleaning no longer blocks the construction of large, clean training datasets for relating drilling responses to rock properties.","feed_headline":"Machine learning beats statistical rules at cleaning drilling data","feed_subtitle":"Isolation forest removes anomalies from rock-drilling streams without tuning, preserving almost all normal data.","key_machinery":"The load-bearing mechanism is the isolation forest anomaly score: each point is isolated by randomly splitting the data on a feature, the average path length $E(h(x))$ to isolate it is compared to the expected path length $c(n)$ for a random tree, and the score $s(x,n)=2^{-E(h(x))/c(n)}$ approaches 1 for points that are 'few and different'. Short paths mean a point is separated quickly, and this rule is what lets IsoForest flag collaring, pause, rod-adding, and end-of-drilling anomalies without tuning. The companion mechanism for mixed holes is two-cluster K-means on standardized features (penetration rate, percussion, feed, rotation pressures): soil drilling points form their own compact cluster in that feature space, so a two-cluster assignment splits soil from rock, and the cluster with distinctly lower percussion pressure and higher penetration rate is removed as soil.","core_discovery":"On the paper's own terms, the central discovery is that anomaly detection in investigative-drilling data is a workable automatic pipeline for rock-dominated holes: isolation forest operating in its default 'auto' mode outperforms conventional statistical cleaning, and outperforms one-class SVM and DBSCAN when the comparison demands no hyperparameter tuning. In the rock-only task, the 3-sigma rule detects 13 of 28 true anomalies, the IQR rule detects all 28 but also removes 34 of 162 normal points, while IsoForest detects 23 of 28 and removes only 4 normal points. In the mixed soil–rock task, IsoForest detects 22 of 26 anomalies with 2 false positives, and after a two-cluster K-means separation it removes 39 of 39 soil points; this is the full automatic cleaning strategy the paper proposes.","pith_inferences":["If the visual labels were instead checked against core logs or down-hole camera images, some points now labeled 'normal'—particularly thin weathered zones or rock seams—might turn out to be informative transitions; an independent physical ground truth would test whether the pipeline removes only operational noise.","The two-stage structure of score-based anomaly removal followed by cluster-based removal of a physically distinct segment is a generic recipe that should transfer to other measurement-while-drilling modalities where one segment class differs sharply, such as soft overburden versus hard rock.","A stronger test of the paper's claim would be to run auto-mode IsoForest across many boreholes and compare each borehole's optimal one-class SVM and DBSCAN settings; if those settings cluster tightly rather than varying, the practical advantage of IsoForest would shrink.","Soil-dominated sites are explicitly left for future work; an immediate testable extension is the mirrored pipeline—two-cluster K-means first to remove soil, then IsoForest on the remaining data—and a check of whether recall and false-positive rates degrade."],"forward_implications":["Cleaning hundreds of investigative-drilling boreholes can be automated with auto-mode IsoForest, removing the labor bottleneck that currently limits large measurement-while-drilling datasets.","For sites with substantial soil cover, the IsoForest plus two-cluster K-means pipeline removes both operational anomalies and the soil section in one pass, with no manual hyperparameter selection.","One-dimensional statistical outlier rules such as the 3-sigma rule and IQR method are shown to be inadequate for multi-feature drilling streams: they either miss anomalies or over-clean normal data that encodes rock-property variation.","Because IsoForest's per-borehole runtime is around $10^{-2}$ seconds, even projects with thousands of boreholes remain computationally feasible.","Per-borehole hyperparameter variability of one-class SVM and DBSCAN would make them impractical to automate across many boreholes unless a tuning-free variant is found."],"supporting_citations":[{"why":"Defines isolation-based anomaly detection and the path-length anomaly score that IsoForest uses, the algorithm the paper finds best.","marker":"Liu et al., 2008, 2012"},{"why":"Supplies the one-class SVM novelty-detection objective whose nu parameter had to be tuned in the comparison.","marker":"Schölkopf et al., 1999"},{"why":"Defines DBSCAN's density-based clustering and noise label, used as the second machine-learning comparator.","marker":"Ester et al., 1996"},{"why":"Provides the Scikit-learn implementations of IsoForest, one-class SVM, DBSCAN, K-means, and StandardScaler used in the experiments.","marker":"Pedregosa et al., 2011"},{"why":"Supports the paper's explanation that tree-based algorithms are comparatively insensitive to hyperparameter settings.","marker":"Probst et al., 2019"}],"fun_headline_variants":["Isolation forest cleans drilling data without tuning, beats statistics","ML automatic cleaning of drilling data outperforms 3-sigma and IQR","Isolation forest: best auto-cleaner for rock drilling data streams","Unsupervised ML removes anomalies from drilling data, preserving normals","IsoForest automatic cleaning beats manual stats in drilling studies"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Every performance figure is measured against manually assigned anomaly and soil labels for two boreholes, made by visual inspection of depth profiles rather than independent physical measurement; if those labels are inexact or the two holes are unrepresentative, the ranking of the algorithms need not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Isolation forest cleans drilling data without tuning, beats statistics","ML automatic cleaning of drilling data outperforms 3-sigma and IQR","Isolation forest: best auto-cleaner for rock drilling data streams","Unsupervised ML removes anomalies from drilling data, preserving normals","IsoForest automatic cleaning beats manual stats in drilling studies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000533,"raw_usage":{"total_tokens":2579,"prompt_tokens":976,"completion_tokens":1603,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":592,"completion_tokens_details":{"reasoning_tokens":1513}},"tokens_in":592,"tokens_out":1603,"duration_ms":10795,"temperature":1.0,"reasoning_tokens":1513,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T19:53:12.929143+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same five cleaning methods on a third borehole whose anomalies are identified by an independent source such as core recovery logs, a drilling diary, or a down-hole camera, and compare recall and false positives; if auto-mode IsoForest's recall drops well below the 23/28 figure reported here, or if a per-borehole tuned SVM or DBSCAN consistently beats it, the paper's central ranking is not stable.","supporting_citations":[],"review_version":1}