Pith. sign in

REVIEW 4 major objections 6 minor 5 references

Machine learning approaches for automatic cleaning of investigative drilling data

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Machine learning beats statistical rules at cleaning investigative-drilling data.

desk verdict A legitimate first application of off-the-shelf anomaly detectors to investigative drilling data, but the 'IsoForest is best' claim rests on two manually labeled boreholes and margins of one or two points; the paper deserves peer review but needs a much more careful evaluation and a softer conclusion. read the letter →

arxiv 2506.14289 v1 pith:VNOPGB5K submitted 2025-06-17 physics.geo-ph physics.data-an

Fei Huang (1) , Hongyu Qin (1) , Masoud Manafi (2) , Ben Juett (2) , Ben Evans (2) ((1) Flinders University , (2) Civil Group (Aust) Pty Ltd) This is my paper · ORCID
classification physics.geo-phphysics.data-an
keywords investigativedrillingmeasurementwhiledatacleaninganomalydetectionisolationforestone-classSVMDBSCANrock
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Investigative drilling produces a near-continuous depth record of penetration rate and percussion, feed, and rotation pressures, but non-informative stretches—drill collaring, unexpected pauses, rod adding, end-of-drilling, and any soil section in a rock hole—contaminate the stream. The paper asks whether unsupervised machine learning can remove those stretches automatically, and claims the answer is yes: on two example boreholes, isolation forest, one-class SVM, and DBSCAN all beat the standard 3-sigma and interquartile-range rules at detecting anomalies while keeping normal data. Isolation forest is singled out as the best of the three because it reaches 23 of 28 anomalies and only 4 false positives on the rock-only borehole with default settings, whereas the other two algorithms need per-borehole hyperparameter tuning. For mixed soil–rock holes, the paper adds a two-cluster K-means step after isolation forest, which separates and removes all soil drilling data without additional false positives. The broader point is practical: if these results hold across many boreholes, manual data cleaning no longer blocks the construction of large, clean training datasets for relating drilling responses to rock properties.

What carries the argument

The load-bearing mechanism is the isolation forest anomaly score: each point is isolated by randomly splitting the data on a feature, the average path length $E(h(x))$ to isolate it is compared to the expected path length $c(n)$ for a random tree, and the score $s(x,n)=2^{-E(h(x))/c(n)}$ approaches 1 for points that are 'few and different'. Short paths mean a point is separated quickly, and this rule is what lets IsoForest flag collaring, pause, rod-adding, and end-of-drilling anomalies without tuning. The companion mechanism for mixed holes is two-cluster K-means on standardized features (penetration rate, percussion, feed, rotation pressures): soil drilling points form their own compact cluster in that feature space, so a two-cluster assignment splits soil from rock, and the cluster with distinctly lower percussion pressure and higher penetration rate is removed as soil.

What would settle it

Run the same five cleaning methods on a third borehole whose anomalies are identified by an independent source such as core recovery logs, a drilling diary, or a down-hole camera, and compare recall and false positives; if auto-mode IsoForest's recall drops well below the 23/28 figure reported here, or if a per-borehole tuned SVM or DBSCAN consistently beats it, the paper's central ranking is not stable.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that anomaly detection in investigative-drilling data is a workable automatic pipeline for rock-dominated holes: isolation forest operating in its default 'auto' mode outperforms conventional statistical cleaning, and outperforms one-class SVM and DBSCAN when the comparison demands no hyperparameter tuning. In the rock-only task, the 3-sigma rule detects 13 of 28 true anomalies, the IQR rule detects all 28 but also removes 34 of 162 normal points, while IsoForest detects 23 of 28 and removes only 4 normal points. In the mixed soil–rock task, IsoForest detects 22 of 26 anomalies with 2 false positives, and after a two-cluster K-means separation it removes 39 of 39 soil points; this is the full automatic cleaning strategy the paper proposes.

Load-bearing premise

Every performance figure is measured against manually assigned anomaly and soil labels for two boreholes, made by visual inspection of depth profiles rather than independent physical measurement; if those labels are inexact or the two holes are unrepresentative, the ranking of the algorithms need not transfer.

Editorial extensions

If this is right

  • Cleaning hundreds of investigative-drilling boreholes can be automated with auto-mode IsoForest, removing the labor bottleneck that currently limits large measurement-while-drilling datasets.
  • For sites with substantial soil cover, the IsoForest plus two-cluster K-means pipeline removes both operational anomalies and the soil section in one pass, with no manual hyperparameter selection.
  • One-dimensional statistical outlier rules such as the 3-sigma rule and IQR method are shown to be inadequate for multi-feature drilling streams: they either miss anomalies or over-clean normal data that encodes rock-property variation.
  • Because IsoForest's per-borehole runtime is around $10^{-2}$ seconds, even projects with thousands of boreholes remain computationally feasible.
  • Per-borehole hyperparameter variability of one-class SVM and DBSCAN would make them impractical to automate across many boreholes unless a tuning-free variant is found.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the visual labels were instead checked against core logs or down-hole camera images, some points now labeled 'normal'—particularly thin weathered zones or rock seams—might turn out to be informative transitions; an independent physical ground truth would test whether the pipeline removes only operational noise.
  • The two-stage structure of score-based anomaly removal followed by cluster-based removal of a physically distinct segment is a generic recipe that should transfer to other measurement-while-drilling modalities where one segment class differs sharply, such as soft overburden versus hard rock.
  • A stronger test of the paper's claim would be to run auto-mode IsoForest across many boreholes and compare each borehole's optimal one-class SVM and DBSCAN settings; if those settings cluster tightly rather than varying, the practical advantage of IsoForest would shrink.
  • Soil-dominated sites are explicitly left for future work; an immediate testable extension is the mirrored pipeline—two-cluster K-means first to remove soil, then IsoForest on the remaining data—and a check of whether recall and false-positive rates degrade.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. This paper investigates automatic cleaning of investigative drilling (ID) data using three unsupervised anomaly-detection algorithms (IsoForest, one-class SVM, DBSCAN) and compares them with the 3σ rule and IQR method on two manually labeled boreholes. For rock-only borehole A, the task is anomaly removal; for mixed soil/rock borehole B, the paper proposes a hybrid IsoForest + two-cluster K-means strategy to remove both anomalies and soil drilling data. The paper reports confusion-matrix counts and runtimes, concluding that all three machine learning methods outperform the statistical baselines, that IsoForest performs best in auto mode without hyperparameter tuning, and that the hybrid strategy enables fully automatic cleaning of rock-dominated ID datasets.

Significance. If the claims held, the paper would offer a practical, automatic cleaning pipeline for rock-dominated investigative drilling data, supporting construction of large ML-ready geotechnical datasets. The study has clear strengths: it carefully enumerates four types of drilling-action anomalies; it uses standard, reproducible scikit-learn implementations; it reports detailed confusion counts per anomaly source; and it tests the hybrid method on a second borehole to check over-cleaning. The main limitation is the evidence base: all quantitative conclusions rely on only two manually labeled boreholes, with no held-out data, no cross-validation, and no uncertainty quantification. The contribution is therefore best viewed as a promising proof-of-concept rather than a definitive comparative evaluation.

major comments (4)
  1. [Section 4.1, Tables 3–4; Section 2, Table 2] The central claim that IsoForest is the best-performing algorithm and that all ML methods outperform statistical methods rests on only two boreholes with manually assigned labels. The reported metrics are computed on the same data used for visual labeling and hyperparameter selection. The numerical ranking among algorithms is fragile: in borehole A, DBSCAN detects 26/28 anomalies versus IsoForest's 23/28, with 6 versus 4 false positives; in borehole B, DBSCAN detects 21/26 versus IsoForest's 22/26 with identical false positives. With n=2 and no confidence intervals or resampling, the abstract's statement that IsoForest is 'proven to be the best-performing algorithm' is not supported. The authors should either validate on additional boreholes with label reliability assessment or substantially temper the claims in the abstract and conclusions.
  2. [Section 4.1, one-class SVM and DBSCAN tuning; Section 4.2] One-class SVM (nu) and DBSCAN (eps, min_samples) were tuned by trial and error on the same boreholes to match IsoForest's rod-adding baseline, and their final scores are reported on those same boreholes. This makes the comparison favorable to IsoForest by construction: the competing methods are hand-fit to the test data while IsoForest runs in auto mode. The statement in the abstract that one-class SVM and DBSCAN 'required hyperparameter tuning' is accurate, but the subsequent claim that IsoForest is superior for automatic cleaning is not a fair out-of-the-box comparison. A proper evaluation would use a separate validation split for tuning or default hyperparameters for all algorithms, and would report performance on held-out boreholes.
  3. [Section 2, Table 2] The anomaly labels were created by visual inspection of depth profiles without an independent physical measurement or inter-rater reliability check. Since every recall and false-positive count in Tables 3 and 4 is defined relative to these labels, the evaluation inherits the subjectivity of the labeling. This is consequential because some labeled anomalies (e.g., collaring onset, rod-adding) are contiguous with normal transitions; a one-point shift in labeling could alter the ranking between IsoForest and DBSCAN in either borehole. The authors should describe the labeling protocol in more detail, ideally with independent labels from a second expert or with quantitative criteria based on the known operational actions.
  4. [Tables 3–4 and Figure 10] The paper does not provide a single aggregate performance metric (e.g., F1-score, Matthews correlation coefficient, or a cost-weighted score) or a significance test to support the statement that 'all three machine learning algorithms outperformed traditional statistical methods' in both tasks. For example, in borehole A the IQR method achieves 100% recall but misclassifies 34 of 162 normal points, while IsoForest has 82% recall and only 4 false positives; whether this constitutes 'outperforming' depends on the relative cost of false positives and false negatives. Without an explicit criterion or metric, the superiority claim is a qualitative judgment rather than a demonstrated result.
minor comments (6)
  1. [Section 4.1, DBSCAN paragraph] The phrase 'tuned to the baseline performance of IsoForest detailed forehead' contains a typo; 'forehead' should be 'aforementioned' or 'above'.
  2. [Section 3.2.1, Eq. (3)] Equation (3) is garbled in the text; it should read approximately c(n) = 2H(n−1) − 2(n−1)/n. Please correct the typesetting.
  3. [Abstract and Section 4.1] The 'auto mode' of IsoForest is not explicitly defined. It would be helpful to state which scikit-learn parameters (e.g., contamination, n_estimators, max_samples) are left at defaults, so readers can reproduce the 'no hyperparameter adjustment' claim.
  4. [Table 4, last row] For the 'IsoForest combined with two-cluster K-means' row, the hyperparameter column says 'None', but K-means uses k=2. List the full parameter set for reproducibility.
  5. [Figure 10] The normalized confusion matrices should specify whether rows and columns correspond to actual or predicted labels, and the text's reference to the 'bottom-right quadrant' for true positive rate should be checked against the figure's orientation.
  6. [Throughout] The spelling of 'one-class SVM' is inconsistent ('one-class SVM', 'one-Class SVM'); please unify. Also, the abstract phrase 'hyperparameter tuning s were required' has an agreement error.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper is an empirical benchmark whose claims depend on labeled evaluation data and tuned baselines, but none of the reported results is equivalent to its inputs by construction.

full rationale

The paper makes no first-principles derivation; it reports an empirical algorithm comparison. The manual anomaly labels in Table 2 are a ground-truth evaluation set, not an input to IsoForest, and IsoForest's auto-mode outputs are computed without using the labels, so the recall and false-positive rates are not self-fulfilling. One-class SVM and DBSCAN hyperparameters were tuned on the same two boreholes to match IsoForest's rod-adding behavior (Sections 4.1 and 4.2), which weakens the comparison as an estimate of out-of-borehole performance, but this is a validation-design limitation rather than circularity: the tuning does not force IsoForest's final counts or ranking, and the reported differences (e.g., 23/28 vs 26/28 anomalies and 4 vs 6 false positives in borehole A) are contingent outputs, not identities. The K-means hybrid step is unsupervised and not fitted to the manual labels. The only self-citations, such as Huang et al. (2025), support background claims about ID noise reduction and site-characterisation potential; they are not load-bearing for the cleaning comparison. No equation or metric is defined in terms of the quantity it is used to predict, so there is no circular step to report.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The paper is an empirical comparison, so it introduces no physical entities. Its free parameters are the tuned hyperparameters of one-class SVM and DBSCAN plus the K-means cluster count. The axioms are the algorithm's distributional assumption, the correctness of manual labels, the two-cluster separability of soil and rock, the representativeness of two boreholes, and the correctness of the scikit-learn implementations.

free parameters (5)
  • one-class SVM nu (borehole A) = 0.2
    Chosen by trial and error to match IsoForest's rod-adding baseline in Section 4.1; controls the outlier fraction and materially changes detection results.
  • one-class SVM nu (borehole B) = 0.3
    Tuned separately in Section 4.2 to match IsoForest's rod-adding detection; differs from the borehole A value, showing dataset dependence.
  • DBSCAN eps and min_samples (borehole A) = eps=0.7, min_samples=10
    Trial-and-error tuning in Section 4.1; the number of detected anomalies is sensitive to eps.
  • DBSCAN eps and min_samples (borehole B) = eps=0.35, min_samples=30
    Tuned separately in Section 4.2; differs from the borehole A values, which the paper cites as a barrier to automation.
  • K-means number of clusters k = 2
    Chosen to separate soil from rock by the assumption that the two geomaterials form two clusters; the choice is not validated against other values of k.
assumptions (5)
  • domain assumption Anomalies in ID data are rare and distinct, the core 'few and different' premise of isolation forest.
    Invoked in Section 3.2.1 via Liu et al. (2008). If ID anomalies are not separable by short isolation paths, auto-mode IsoForest would fail; only two boreholes test this.
  • domain assumption The manual labels for collaring, unexpected pauses, rod adding, end of drilling, soil-rock transitions, and soil drilling are correct ground truth.
    Section 2 and Table 2. All performance metrics compare against these labels, so any label error directly biases recall and false positive rate.
  • domain assumption Soil and rock drilling data occupy two well-separated clusters in the scaled four-feature space.
    Section 4.2 uses two-cluster K-means after IsoForest. If multiple strata or gradational transitions produce more than two clusters, the hybrid would either retain soil or delete rock data.
  • domain assumption The two example boreholes are representative of the broader population of ID boreholes.
    Borehole A has 190 rows and borehole B has 183 rows (Section 2). The conclusion that IsoForest needs no tuning across hundreds of boreholes generalizes only if these two capture typical data distributions.
  • standard math The scikit-learn implementations of IsoForest, one-class SVM, DBSCAN and K-means behave as documented.
    Section 3.2 relies on these implementations via Pedregosa et al. (2011); any library bug or default behavior change would alter the reported results.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Machine learning approaches for automatic cleaning of investigative drilling data." pith.science (2026). https://pith.science/paper/VNOPGB5K

@misc{pith2026250614289,
  author       = {Pith},
  title        = {Pith review of: Machine learning approaches for automatic cleaning of investigative drilling data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VNOPGB5K}},
  note         = {Machine review of arXiv:2506.14289}
}
read the original abstract

Investigative drilling (ID) is an innovative measurement while drilling (MWD) technique that has been implemented in various site investigation projects across Australia. While the automated drilling feature of ID substantially reduces noise within drilling data streams, data cleaning remains essential for removing anomalies to enable accurate strata classification and prediction of soil and rock properties. This study employed three machine learning algorithms--IsoForest, one-class SVM, and DBSCAN--to automate the data cleaning process for ID data in rock drilling scenarios. Two data cleaning contexts were examined: (1) removing anomalies in rock drilling data, and (2) removing both anomalies and soil drilling data in mixed rock drilling data. The analysis revealed that all three machine learning algorithms outperformed traditional statistical methods (the 3-sigma rule and IQR method) in both data cleaning tasks, achieving a good balance between true positive rate and false positive rate, though hyperparameter tuning was required for one-class SVM and DBSCAN. Among them, IsoForest was proven to be the best-performing algorithm, capable of removing anomalies effectively without the need for hyperparameter adjustment. Furthermore, IsoForest, combined with two-cluster K-means, successfully eliminated both soil drilling data and anomalies while preserving almost all the normal data. The automatic data cleaning strategy proposed in this paper has the potential to reduce laborious manual data cleaning efforts and thereby facilitate the development of large-scale, high-quality datasets for machine learning studies capable of revealing complex relationships between drilling data and rock properties.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

5 extracted references · 4 canonical work pages

  1. [1]

    Chen, J., & Yue, Z. Q. (2015). Ground characterization using breaking -action-based zoning analysis of rotary- percussive instrumented drilling. International Journal of Rock Mechanics and Mining Sciences, 75, 33-43. https://doi.org/10.1016/j.ijrmms.2014.11.008 Chen, L. H., & Labuz, J. F. (2006). Indentation of rock by wedge -shaped tools. International J...

    arXiv 2015

  2. [2]

    Q., Lee, C

    Yue, Z. Q., Lee, C. F., Law, K. T., & Tham, L. G. (2004). Automatic monitoring of rotary -percussive drilling for ground characterization—illustrated by a case example in Hong Kong. International Journal of Rock Mechanics and Mining Sciences, 41(4), 573-612. https://doi.org/10.1016/j.ijrmms.2003.12.151

    work page doi:10.1016/j.ijrmms.2003.12.151 2004

  3. [146]

    https://doi.org/10.1016/j.ijrmms.2015.09.004 Kalantari, S., Hashemolhosseini, H., & Baghbanan, A. (2018). Estimating rock strength parameters using drilling data. International Journal of Rock Mechanics and Mining Sciences , 104, 45 -52. https://doi.org/10.1016/j.ijrmms.2018.02.013 Lee, H., & Lee, H. P. (2023). Formation lithology predictions based on mea...

    work page doi:10.1016/j.ijrmms.2015.09.004 2018

  4. [227]

    https://doi.org/10.1016/j.geoen.2023.211917 Leung, R., & Scheding, S. (2015). Automated coal seam detection using a modulated specific energy measure in a monitor-while-drilling context. International Journal of Rock Mechanics and Mining Sciences, 75, 196-209. Li, Z., & Itakura, K. (2012). An analytical drilling model of drag bits for evaluation of rock s...

    work page arXiv 2015

  5. [4565]

    W., Soga, K., Bolton, M

    https://doi.org/10.3390/app12094565 Gui, M. W., Soga, K., Bolton, M. D., & Hamelin, J. P. (2002). Instrumented Borehole Drilling for Subsurface Investigation. Journal of Geotechnical and Geoenvironmental Engineering , 128(4), 283 -291. https://doi.org/10.1061/(asce)1090-0241(2002)128:4(283) Hansen, T. F., Erharter, G. H., Liu, Z., & Torresen, J. (2024). A...

    work page doi:10.3390/app12094565 2002

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.