Pith. sign in

REVIEW 2 major objections 6 minor 28 references

Physics-Informed Machine Learning Under Small-Data Constraints: Lessons from Abrasive Waterjet Milling

T0 review · 2 major / 6 minor · reviewed 2026-07-10 · grok-4.5

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read On small process datasets, curation choices, evaluation protocol, and how physics enters the model can swing rankings as much as the algorithm.

desk verdict Solid methodological checklist paper on a real n=155 AWJM set: curation-as-hypothesis, single-split rank reversal, and algorithm-dependent residual physics are cleanly shown and carefully scoped. read the letter →

arxiv 2607.07863 v1 pith:TQLWIRFO submitted 2026-07-08 cs.LG

Sarah Grewe , Jörg Frochte This is my paper
classification cs.LG
keywords Small-datamachinelearningPhysics-informedmodellingGaussianprocessesAbrasivewaterjetmillingProcessDatacurationResidualUncertaintycalibration
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that when experimental process data are small, expensive, and material-specific, the choices made before and around the learner matter as much as which learner you pick. Using 155 abrasive waterjet milling runs on Inconel 718, the authors separate physics-based cleaning of clear errors from statistical curation of extreme but plausible points, and treat the latter as competing modelling hypotheses rather than silent preprocessing. They show that a 15-point hold-out ranking is unstable under 10-fold cross-validation, with the single-split winner falling to rank 7 while Gaussian process variants take the top ranks. Residual learning on a compact physics backbone is competitive for GPs and yields lower fold variance plus an interpretable physics-plus-residual decomposition, but degrades tree-based models. Bayesian hyperparameter search helps parameter-sensitive baselines yet harms multi-stage hybrid pipelines at this sample size, and standalone GP intervals are only approximately calibrated. The practical message is that credible comparison on such datasets requires explicit curation hypotheses, resampling-based evaluation, and physics integration matched to the learner's inductive bias.

What carries the argument

Four physics-integration levels (pure ML features; log-transformed features; residual learning on the four-parameter PG-H(4) physics backbone; PG-H(4) as GP prior mean) crossed with competing Stage-2 curation hypotheses (none, MAD thresholds, IQR fence) and single-split versus 10-fold evaluation.

What would settle it

Replicate the same Stage-1 cleaning, Stage-2 curation hypotheses, four physics-integration levels, and single-split versus 10-fold protocol on at least one additional material or machining process of comparable size; if residual physics no longer helps GPs, trees no longer degrade, and single-split rankings no longer reverse under multi-fold CV, the claimed methodological picture fails.

Watch

Extended reading notes

Core claim

For small, expensive, material-specific process datasets, reliable model comparison depends on treating statistical curation as an explicit modelling hypothesis, using multi-fold rather than single-split evaluation, and matching the form of physics integration to the learner: residual learning on a compact physics baseline helps Gaussian processes (lower variance, interpretable decomposition) but degrades tree-based methods, while Bayesian tuning helps simple baselines yet can harm multi-stage hybrids at n around 100.

Load-bearing premise

The methodological lessons drawn from one material, one machine, and one laboratory are assumed to transfer to the wider class of small-n precision-manufacturing problems because those problems share the same structural traits of tiny samples, uneven data quality, imperfect physics, and evaluation sets too small for stable ranking.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 6 minor

Summary. The paper examines methodological choices for physics-informed machine learning on small, expensive process datasets, using n=155 abrasive waterjet milling experiments on Inconel 718. It advances three claims: (1) physics-based cleaning should be separated from statistical curation, with the latter treated as competing modelling hypotheses rather than silent preprocessing; (2) model rankings from a 15-point hold-out are unstable relative to 10-fold CV (the single-split winner falls from rank 1 to 7), with GP variants occupying the top ranks under CV; (3) residual learning on a compact four-parameter physics baseline (PG-H(4)) is competitive for GPs—lower fold variance and an interpretable physics/residual decomposition—but degrades tree-based models, while Bayesian hyperparameter tuning helps parameter-sensitive baselines (GB, SVR) yet harms multi-stage hybrid pipelines at this sample size. GP predictive intervals are approximately calibrated (86% empirical coverage at nominal 90%). The methodological conclusion is carefully scoped: for small process datasets of this kind, reliable comparison benefits from explicit curation hypotheses, resampling-based evaluation, and physics integration matched to the learner’s inductive bias.

Significance. If the structural lessons transfer beyond this single material and machine, the paper is a useful methodological contribution for manufacturing process modelling and other small-n scientific regression settings. Strengths include a clean evaluation design (Stage-2 filters recomputed inside each CV fold from training data only; no leakage), honest reporting of rank reversal (Table 2), algorithm-dependent physics effects (Fig. 1, §5.3), low-power statistical tests, and an interpretable residual decomposition. The work does not claim universal algorithm superiority; conclusions are scoped to structural properties (small n, heterogeneous quality, imperfect physics, tiny evaluation sets). The main limitation is single-dataset external validity, already flagged in §6.1. Within that scope the evidence is coherent and actionable for practitioners facing similar constraints.

major comments (2)
  1. [§4.2 and §5.3] §4.2 defines four physics integration levels, including Level 3 (PG-H(4) as GP prior mean), described as “the strongest integration but most sensitive to physics model accuracy.” Results in §5 report only Levels 0–2 (and L1 only in Fig. 1). Without Level-3 numbers, the claimed “spectrum” study is incomplete at the strongest integration point, and the reader cannot assess whether reversion to the physics prior helps or hurts relative to residual learning. Either report Level 3 under the same protocol or narrow the claim to the levels actually evaluated.
  2. [§5.3] §5.3 reports a training-size ablation in which the hybrid GP’s fold-to-fold variance advantage over pure GP appears only at full training size (ratio 1.6:1), while at 20–60% the ratio is ≤1.2:1. This is reported honestly but sits in tension with the small-data motivation for residual physics. The discussion should state more clearly what this implies for the recommended use of residual learning when n is even smaller than the present full set, rather than leaving the implication implicit.
minor comments (6)
  1. [Abstract] Abstract and elsewhere: “hyper parameter” should be “hyperparameter” (standard compound form).
  2. [§4.4 / Fig. 1] MAPE is used as the primary visual metric in Fig. 1 and discussed in §5.3, but is not introduced in §4.4 (Evaluation Protocol), which only defines RMSE and MAE. Add a brief definition and justification for including MAPE.
  3. [§4.1 / References] Author name encoding for Hlaváč appears inconsistently (Hlav’aˇc / Hlaváˇc) in the text and references; normalise to a single Unicode form.
  4. [Table 2] Table 2 caption is dense; a short note that ranks are by 10-fold RMSE and that L0/L2 refer to §4.2 would improve readability without requiring the main text.
  5. [§5.5] §5.5: the delta-method back-propagation σ_y ≈ 2|y_t|σ_{y_t} is stated without a short derivation or reference; one sentence would help readers who do not immediately recall the transform Jacobian.
  6. [§4.3] Four additional tree-based methods are said to rank below all GP and standard GB variants under 10-fold CV but are “reported only in aggregate” (§4.3). A one-row summary in a supplement or appendix would make that claim checkable.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical small-data ML comparison with proper hold-out/CV evaluation; physics baseline is a fitted backbone, not a self-defining prediction.

full rationale

The paper is a controlled empirical study of curation hypotheses, evaluation protocols, and physics-integration levels on a fixed n=155 AWJM dataset. All reported rankings, RMSE/MAE, variance ratios, and coverage figures are obtained from single-split hold-out or 10-fold CV with Stage-2 filters recomputed only on training folds (explicit anti-leakage). The compact PG-H(4) physics model (Eq. 1) is fitted by nonlinear least squares on training data and then used either as a residual target or GP prior mean; residual learning is evaluated on held-out folds, not reverse-engineered from the final metric. No quantity claimed as a “prediction” or “first-principles result” reduces by construction to a fitted constant or to the evaluation statistic itself. Citations are ordinary background (Hashish, Hlaváč, Rasmussen & Williams, Optuna, etc.); none supply a uniqueness theorem or load-bearing ansatz that the present claims rest upon. The methodological conclusions are scoped to the observed structural properties of this dataset and are therefore self-contained against the paper’s own experimental protocol. Score 0 is the honest finding.

Assumptions & free parameters 4 free parameters · 4 assumptions · 1 invented entities

The central methodological claims rest on a compact physics backbone whose free parameters are fitted to the same small dataset, on hand-chosen statistical thresholds treated as hypotheses, and on standard domain assumptions about jet physics. No new physical entities are postulated; the ledger is therefore short and mostly consists of fitted scalars and modelling choices made explicit by the authors.

free parameters (4)
  • PG-H(4) coefficients C, A, B, γ
    Four free parameters of the rational power-law baseline fitted by nonlinear least squares (soft-L1, 10 restarts) on the training partition; they define the residual that Level-2/3 models learn.
  • MAD thresholds τ = 2.5 and 3.5
    Hand-chosen cut-offs that define two of the four competing curation hypotheses; downstream RMSE gaps of >0.10 mm for GPs depend on these values.
  • IQR fence multiplier 1.5
    Standard Tukey constant used for H-IQR; again a modelling choice whose effect is measured rather than derived.
  • GP Matérn-5/2 length-scales and noise variance
    Optimised by marginal likelihood; affect ranking and calibration numbers that support the claim that GPs occupy the top ranks.
assumptions (4)
  • domain assumption Jet kinetic energy scales as p^{3/2} from Bernoulli’s equation, so the pressure exponent can be fixed at 3/2 while γ remains free.
    Invoked in Section 4.1 to justify the functional form of PG-H(4); standard fluid-dynamics premise, not re-derived.
  • ad hoc to paper A compact four-parameter physics model that captures only ~half the variance is still useful as a residual backbone for smooth interpolators.
    Stated in Section 4.1–4.2; the claim that residual learning is competitive for GPs rests on this modelling premise.
  • domain assumption Stage-1 physics cleaning (sign anomalies, variation coefficient >1, glazing regime) removes only demonstrably erroneous points and can be applied unconditionally.
    Section 3.2; all subsequent n=130 analyses assume these 25 points are correctly discarded.
  • standard math 10-fold CV on the always-clean pool plus filter-augmented training folds yields more stable rankings than a 15-point hold-out.
    Standard statistical premise, supported by cited literature (Vabalas, Varoquaux) and by the observed rank reversal.
invented entities (1)
  • PG-H(4) rational physics baseline
    purpose: Compact four-parameter backbone that supplies the residual target for Level-2/3 hybrids and the prior mean for Level 3.
    Constructed by the authors from Hashish/Hlaváč scaling plus an ad-hoc linear denominator; no independent experimental confirmation outside this dataset is provided.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Physics-Informed Machine Learning Under Small-Data Constraints: Lessons from Abrasive Waterjet Milling." pith.science (2026). https://pith.science/paper/TQLWIRFO

@misc{pith2026260707863,
  author       = {Pith},
  title        = {Pith review of: Physics-Informed Machine Learning Under Small-Data Constraints: Lessons from Abrasive Waterjet Milling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TQLWIRFO}},
  note         = {Machine review of arXiv:2607.07863}
}
abstract

In physically dominated machining processes, experimental datasets are small, expensive, and material-specific; in this regime, data curation, evaluation design, and the form of physics integration can matter as much as the learning algorithm. Using an abrasive waterjet milling dataset ($n{=}155$, Inconel\,718), we make three methodological contributions. First, we separate physics-based data \emph{cleaning} from statistical \emph{curation} and treat the latter as competing modelling hypotheses rather than silent preprocessing. Second, we find that model rankings from a 15-point hold-out set can be unstable: the single-split winner drops from rank~1 to rank~7 under 10-fold cross-validation, while Gaussian Process (GP) variants occupy the top ranks. Third, we study a spectrum of physics integration levels and find that residual learning on a compact physics baseline is competitive for GP, yielding lower variance and an interpretable decomposition, but degrades tree-based models. Bayesian hyper parameter tuning improves parameter-sensitive baselines such as gradient boosting and SVR, yet harms multi-stage hybrid pipelines at this sample size. GP uncertainty intervals are approximately calibrated ($86\%$ empirical coverage at nominal $90\%$). The resulting picture is methodological: for small, expensive process datasets, our results suggest that, in this setting, reliable model comparison benefits from explicit curation hypotheses, robust evaluation, and careful choices about how physics enters the model.

Figures

Figures reproduced from arXiv: 2607.07863 by the authors.

Figure 1
Figure 1. Physics integration effect on MAPE across 10 folds. For GP, Level 2 is competitive (similar median, lower [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Depth-dependent uncertainty quality. Left: 90%-CI coverage by depth bin; the dashed line marks the nominal 90% level. GP L0 maintains near-nominal coverage for shallow and medium cuts; GP L2 degrades sharply at 0.3–1.0 mm. Right: Mean 90%-CI width. The hybrid produces wider but less reliable intervals at medium and deep cuts. that these global figures average over distinct regimes. At shallow cuts (| ¯d| < 0.3 mm) b… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

28 extracted references · 28 canonical work pages

  1. [1]

    In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining

    Akiba, T., Sano, S., Yanase, T., Ohta, T., Koyama, M.: Optuna: A next-generation hyperparameter optimization framework. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. pp. 2623–2631 (2019). https://doi.org/10.1145/3292500.3330701

    work page doi:10.1145/3292500.3330701 2019

  2. [2]

    N., Etxeberria, I., Suárez, A.: Effect of process parameter on the kerf geometry in abrasive water jet milling

    Alberdi, A., Rivero, A., López de Lacalle, L. N., Etxeberria, I., Suárez, A.: Effect of process parameter on the kerf geometry in abrasive water jet milling. The International Journal of Advanced Manufacturing Technology 51(5–8), 467–480 (2010). https://doi.org/10.1007/s00170-010-2662-y

    work page doi:10.1007/s00170-010-2662-y 2010

  3. [3]

    Angelopoulos and Stephen Bates

    Angelopoulos, A.N., Bates, S.: Conformal prediction: A gentle introduction. Foundations and Trends in Machine Learning16(4), 494–591 (2023). https://doi.org/10.1561/2200000101

    work page doi:10.1561/2200000101 2023

  4. [4]

    C.: On over-fitting in model selection and subsequent selection bias in performance evaluation

    Cawley, G.C., Talbot, Nicola L. C.: On over-fitting in model selection and subsequent selection bias in performance evaluation. Journal of Machine Learning Research11, 2079–2107 (2010) 7 Physics-Informed ML Under Small-Data ConstraintsA PREPRINT

    work page

  5. [5]

    Journal of Manufacturing Processes110, 291–302 (2024)

    Chen, M., Zhang, S., Lu, G., Wu, Y .: Method of ensemble modeling for abrasive water jet machinability of metal materials. Journal of Manufacturing Processes110, 291–302 (2024). https://doi.org/10.1016/j.jmapro.2024.01.010

    work page doi:10.1016/j.jmapro.2024.01.010 2024

  6. [6]

    Data-Centric Engineering5, e8 (2024)

    Cross, E., Rogers, T., Worden, K.: A spectrum of physics-informed Gaussian processes for regression in engineering. Data-Centric Engineering5, e8 (2024). https://doi.org/10.1017/dce.2024.2

    work page doi:10.1017/dce.2024.2 2024

  7. [7]

    The International Journal of Advanced Manufacturing Technology126, 3133–3148 (2023)

    Deng, H., Yao, P., Hai, K., Yu, S., Huang, C., Zhu, H., Liu, D.: High-efficiency abrasive water jet milling of aspheric RB-SiC surface based on BP neural network depth control models. The International Journal of Advanced Manufacturing Technology126, 3133–3148 (2023). https://doi.org/10.1007/s00170-023-11275-7

    work page doi:10.1007/s00170-023-11275-7 2023

  8. [8]

    https://doi.org/10.1016/j.strusafe.2008.06.020

    Der Kiureghian, A., Ditlevsen, O.: Aleatory or epistemic? Does it matter? Structural Safety31(2), 105–112 (2009). https://doi.org/10.1016/j.strusafe.2008.06.020

    work page doi:10.1016/j.strusafe.2008.06.020 2009

Show all 28 references
  1. [9]

    Journal of Manufacturing Systems73, 898–912 (2024)

    Fullington, D., Yangue, E., Rao, A., Blanco, C., Joo, S.J., Sohn, H., Wicker, R.: Leveraging small-scale datasets for additive manufacturing process monitoring. Journal of Manufacturing Systems73, 898–912 (2024). https://doi.org/10.1016/j.jmsy.2024.04.001

    2024 doi

  2. [10]

    In: Proceedings of the 33rd International Conference on Machine Learning (ICML)

    Gal, Y ., Ghahramani, Z.: Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In: Proceedings of the 33rd International Conference on Machine Learning (ICML). pp. 1050–1059 (2016)

    2016

  3. [11]

    Grinsztajn, L., Oyallon, E., Varoquaux, G.: Why do tree-based models still outperform deep learning on typical tabular data? In: Advances in Neural Information Processing Systems. vol. 35 (2022), datasets and Benchmarks Track

    2022

  4. [12]

    Journal of the American Statistical Association 69(346), 383–393 (1974)

    Hampel, F.R.: The influence curve and its role in robust estimation. Journal of the American Statistical Association 69(346), 383–393 (1974). https://doi.org/10.1080/01621459.1974.10482962

    1974 doi

  5. [13]

    Journal of Engineering Materials and Technology106(1), 88–100 (1984)

    Hashish, M.: A modeling study of metal cutting with abrasive waterjets. Journal of Engineering Materials and Technology106(1), 88–100 (1984). https://doi.org/10.1115/1.3225682

    1984 doi

  6. [14]

    Journal of Engineering Materials and Technology 111(2), 154–162 (1989)

    Hashish, M.: A model for abrasive-waterjet (AWJ) machining. Journal of Engineering Materials and Technology 111(2), 154–162 (1989). https://doi.org/10.1115/1.3226448

    1989 doi

  7. [15]

    Materials14(14), 4032 (2021)

    Hlaváˇc, L.M.: Revised model of abrasive water jet cutting for industrial use. Materials14(14), 4032 (2021). https://doi.org/10.3390/ma14144032

    2021 doi

  8. [16]

    The ASQC Basic References in Quality Control: Statistical Techniques (1993)

    Iglewicz, B., Hoaglin, D.C.: V olume 16: How to detect and handle outliers. The ASQC Basic References in Quality Control: Statistical Techniques (1993)

    1993

  9. [17]

    Journal of Experimental Social Psychology49(4), 764–766 (2013)

    Leys, C., Ley, C., Klein, O., Bernard, P., Licata, L.: Detecting outliers: Do not use standard deviation around the mean, use absolute deviation around the median. Journal of Experimental Social Psychology49(4), 764–766 (2013). https://doi.org/10.1016/j.jesp.2013.03.013

    2013 doi

  10. [18]

    International Journal of Rock Mechanics and Mining Sciences34(1), 17–25 (1997)

    Momber, A.W., Kovacevic, R.: Test parameter analysis in abrasive water jet cutting of rocklike materials. International Journal of Rock Mechanics and Mining Sciences34(1), 17–25 (1997). https://doi.org/10.1016/S1365- 1609(97)80030-4

    1997 doi

  11. [19]

    The International Journal of Advanced Manufacturing Technology119, 503–516 (2021)

    Prabhu, R., Kanthababu, M.: Prediction of surface roughness using machine learning approach for abrasive waterjet milling of alumina ceramic. The International Journal of Advanced Manufacturing Technology119, 503–516 (2021). https://doi.org/10.1007/s00170-021-08052-9

    2021 doi

  12. [20]

    Expert Systems with Applications246, 123168 (2024)

    Prabhu, R., Kanthababu, M.: Prediction of surface roughness and depth of cut in abrasive waterjet milling of alumina ceramic using machine learning algorithms. Expert Systems with Applications246, 123168 (2024). https://doi.org/10.1016/j.eswa.2024.123168

    2024 doi

  13. [21]

    arXiv preprint arXiv:1811.12808 (2018)

    Raschka, S.: Model evaluation, model selection, and algorithm selection in machine learning. arXiv preprint arXiv:1811.12808 (2018)

    2018 arXiv

  14. [22]

    I.: Gaussian Processes for Machine Learning

    Rasmussen, C.E., Williams, Christopher K. I.: Gaussian Processes for Machine Learning. MIT Press, Cambridge, MA (2006)

    2006

  15. [23]

    Information Fusion81, 84–90 (2022)

    Shwartz-Ziv, R., Armon, A.: Tabular data: Deep learning is not all you need. Information Fusion81, 84–90 (2022). https://doi.org/10.1016/j.inffus.2021.11.011

    2022 doi

  16. [24]

    PLOS ONE14(11), e0224365 (2019)

    Vabalas, A., Gowen, E., Poliakoff, E., Casson, A.J.: Machine learning algorithm validation with a limited sample size. PLOS ONE14(11), e0224365 (2019). https://doi.org/10.1371/journal.pone.0224365

    2019 doi

  17. [25]

    NeuroImage180, 68–77 (2018)

    Varoquaux, G.: Cross-validation failure: Small sample sizes lead to large error bars. NeuroImage180, 68–77 (2018). https://doi.org/10.1016/j.neuroimage.2017.06.061

    2018 doi

  18. [26]

    ACM Computing Surveys55(4), 1–37 (2022)

    Willard, J., Jia, X., Xu, S., Steinbach, M., Kumar, V .: Integrating scientific knowledge with ma- chine learning for engineering and environmental systems. ACM Computing Surveys55(4), 1–37 (2022). https://doi.org/10.1145/3514228 8 Physics-Informed ML Under Small-Data Constrai...

    2022 doi

  19. [27]

    npj Computational Materials9, 42 (2023)

    Xu, P., Ji, X., Li, M., Lu, W.: Small data machine learning in materials science. npj Computational Materials9, 42 (2023). https://doi.org/10.1038/s41524-023-01000-z

    2023 doi

  20. [28]

    F., Xie, J., Sun, L.: On the data quality and imbalance in machine learning-based design and manufactur- ing – a systematic review

    Zhao, Y . F., Xie, J., Sun, L.: On the data quality and imbalance in machine learning-based design and manufactur- ing – a systematic review. Engineering45, 105–131 (2024). https://doi.org/10.1016/j.eng.2024.04.024 9

    2024 doi

Pith tools

Reviewed July 10, 2026 · model on record in the stance chip above.