REVIEW 3 major objections 6 minor 52 references
Active Learning via Heteroskedastic Rational Kriging
T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read A heteroskedastic version of rational kriging lets active learning place new design points in regions where the response surface actually varies, yielding surrogates as accurate as deep Gaussian process methods at a fraction of the cost.
desk verdict A solid, fast heteroskedastic emulator with convincing empirical results, but the high-dimensional variance weight estimation needs a closer look. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the heteroskedastic rational kriging model $y(x) = \mu + \tau(x)Z(x)$ with $\tau(x) = \nu/(c_0 + r(x)'c)$, where $Z$ is a stationary Gaussian process and $c_0, r(x), c$ come from the rational kriging predictor. The denominator $c_0 + r(x)'c$ modulates the local variance; fitting $c$ by maximizing the marginal likelihood (with $c'c \le 1$, $c \ge 0$) lets the model learn where the response varies. The active learning criterion is the resulting posterior variance $s^2(x) \propto (1 - r(x)'R^{-1}r(x))/(c_0 + r(x)'c)^2$, whose numerator preserves space-filling behavior and whose denominator directs new points into high-variance regions. The whole algorithm is fast because the correlation matrix and its inverse are computed once before the $c$-optimization.
What would settle it
Run HRK active learning on a synthetic function whose variance is known to be constant, and measure how far the estimated $\tau(x)$ deviates from flat and whether ALM still redistributes points away from uniformity; if the variance function becomes strongly nonconstant on stationary data, the $c$-estimation is overfitting. On a function with a known narrow high-variability band, the method should place a large fraction of new points inside that band.
Extended reading notes
Core claim
The central claim is that rational kriging's variance multiplier $\tau(x) = \nu/(c_0 + r(x)'c)$ already contains enough freedom to model heteroskedasticity, and that estimating the weight vector $c$ jointly with the other hyperparameters turns active learning from a space-filling exercise into a targeted search. The paper derives an empirical-Bayes objective, fixes the lengthscales from an initial rational kriging fit, and optimizes the resulting $(n+1)$-dimensional problem with a gradient method so that $R$ and $R^{-1}$ are computed only once. In the acquisition step, HRK's posterior variance is used as the ALM criterion; the numerator keeps the design spread out while the denominator concentrates points where the modeled variance is high. Experiments on six test functions and two real datasets show comparable or better accuracy than deep Gaussian process surrogates, with a speed advantage that is orders of magnitude.
Load-bearing premise
The load-bearing premise is that the fitted weight vector $c$, constrained only by $c'c \le 1$ and $c \ge 0$, identifies the true local variance from the initial sample instead of overfitting it.
Editorial extensions
If this is right
- With HRK, active learning MacKay (ALM) no longer degenerates into a maximin space-filling design; the method places more runs near sharp transitions, as shown for the underdamped oscillator and Gramacy-Lee function.
- HRK improves both root mean-squared error and interval score over ordinary kriging in nearly every simulation and both real-data studies.
- The method matches or exceeds deep Gaussian process active learning on most test functions while being orders of magnitude faster, making it practical for sequential design.
- Because only point estimates of hyperparameters are needed, the entire active learning loop can be run repeatedly at modest cost.
Reading between the lines
- The same acquisition function could be used inside Bayesian optimization objectives, such as expected improvement, to balance exploration and exploitation; the paper only mentions this direction in its conclusion.
- The $(n+1)$-dimensional $c$ estimate is not regularized beyond the unit-ball and nonnegativity constraints, so the learned variance surface may overfit small initial designs; a simulation with known heteroskedasticity could quantify this.
- The candidate-set approximation via down-sampled Latin hypercubes may miss narrow high-variance ridges in high dimensions; replacing it with adaptive Voronoi-style candidates is a natural stress test.
- The speed comparison applies to point-estimate fitting; the method's uncertainty statements do not propagate uncertainty in $c$, so coverage intervals may be optimistic when the initial sample is small.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Heteroskedastic Rational Kriging (HRK), a Gaussian-process model in which the response variance is modeled as tau(x) = nu / (c0 + r(x)'c), and uses the resulting posterior variance as an active-learning acquisition criterion. The model is estimated by profile maximum likelihood with theta fixed at the rational-kriging estimate, and the acquisition function in Eq. (17) seeks points with high ordinary-kriging variance relative to the estimated local variance scale. The authors compare HRK with ordinary kriging and deep Gaussian processes on six test functions and two real datasets, reporting lower RMSE and interval scores in most settings and computation times orders of magnitude faster than DeepGP.
Significance. If the variance-parameter estimation is stable, HRK is a valuable contribution: it gives a tractable likelihood and closed-form gradient for a nonstationary GP, an efficient sequential-design criterion, and empirical evidence of improved accuracy over OK with a large speed advantage over DeepGP. The paper is also commendable for including two real-data studies and for making the algorithmic steps (Algorithms 1 and 2) explicit. However, no code or data are provided, and the central variance weights c are fit without a penalty, so the paper's main claim rests on empirical demonstrations that do not directly address overfitting of the variance function. The proposed method is novel and the qualitative conclusions are plausible, but the missing stability analysis is load-bearing for the active-learning claim.
major comments (3)
- [Section 3, Eq. (16) and Algorithm 1] The unpenalized maximum-likelihood fit of c, an (n+1)-dimensional vector estimated from n observations with only the constraints c'c <= 1 and c >= 0, is the load-bearing step for the heteroskedastic variance. The statement in Section 3 that initializing at c_RK 'enforces some sort of regularization' is not a statistical penalty; the optimizer is free to move far from the initial value, and the objective in Eq. (16) contains no complexity term. If c overfits the residuals, tau(x) and the acquisition function in Eq. (17) will concentrate design points in spurious high-variance regions, which would undermine the central active-learning claim. The empirical sections do not isolate this failure mode: there are no plots or summaries of the estimated c, no re-randomization stability checks, and no experiments with a known tau(x). Please add either a penalized or otherwise regularized estimator with a consistency or shrinkage analysis, or a diagnostic study showing that c recovers a known variance function and is stable under re-randomization of the initial design.
- [Algorithm 1, Step 2 and Section 3] Theta is fixed at the rational-kriging estimate when optimizing c, and mu is fixed at mu_RK in Eq. (16). Since R and R^{-1} enter both the likelihood and the acquisition criterion, a poor theta estimate can distort the variance function even if c were otherwise well estimated. The paper does not report any sensitivity analysis for this profile approximation. Please justify fixing theta at theta_RK, for example by showing that the active-learning results are stable to a small grid of theta values or to a one-step joint update, or by stating precisely why theta_RK is sufficient for the heteroskedasticity to be captured.
- [Section 5, Figure 7] The abstract claims 'comparable or better performance relative to other non-stationary Gaussian process-based methods', but in Figure 7 DeepGP is run only for a few steps for most test functions, with the full trajectory shown only for the Gramacy-Lee function. The summary curves therefore compare HRK and OK over the full horizon but DeepGP over a shorter one, and the current wording is stronger than what the truncated comparison supports. Please report the exact number of active-learning steps completed for each function, state how the medians are computed when trajectories have different lengths, and qualify the comparison accordingly.
minor comments (6)
- [Section 4, paragraph after Eq. (17)] There is a typo in 'ALM startegy' that should read 'strategy'.
- [Algorithm 1, Step 1] The notation theta_RK, mu_RK, and c_RK is used before being defined; please define these as the rational-kriging estimates from Joseph (2024).
- [Section 4, Eq. (17)] It would help to state explicitly that the constant factor nu^2 is omitted from the argmax in Eq. (17), since the full posterior variance is s^2(x) = nu^2 (1 - r'R^{-1}r) / (c0 + r'c)^2.
- [Section 5, Figure 7] The shaded bands are described as 5th and 95th quantiles; please clarify whether these are across the 10 repetitions or across a different number of replications, and state the number of repetitions in each panel.
- [Section 6.1] The description of the molecular descriptors is too brief to reproduce: please give details on how the RDKit descriptors were computed, any preprocessing or scaling, and how the five principal components were obtained.
- [Global] No code or data links are provided for the simulations or real-data analyses. Given the novelty of the MMA-based optimization and the central role of the estimated c vector, releasing code would substantially strengthen reproducibility.
Circularity Check
No significant circularity; the HRK model, its estimation, and its active-learning evaluation are self-contained and benchmarked externally.
full rationale
The paper's derivation chain is explicit: the heteroskedastic rational kriging model is defined in Section 3 as y(x) = mu + tau(x) Z(x) with tau(x) = nu / (c0 + r(x)'c), the predictive equations (7)-(9) follow from standard GP conditioning, the hyperparameters are estimated by marginal maximum likelihood in (11)-(16), and the active-learning criterion (17) is the fitted posterior variance. None of these steps assumes the conclusion that HRK improves active learning; the variance function is estimated from the data and then used to choose points, which is standard empirical-Bayes practice rather than a definitional equivalence. The central claim of improved accuracy and speed is supported by simulations on independent test functions and two real datasets, not by the self-citation to Joseph (2024). That citation supplies the base rational-kriging model, but the contribution of this paper—the heteroskedastic extension and its sequential design behavior—is evaluated against external benchmarks, so the citation is not load-bearing in a circular way. The high-dimensional estimation of c in Algorithm 1 could raise overfitting or identifiability concerns, but that is a statistical robustness issue, not a circularity, because the acquisition function is not forced to match the fitted variance by construction; it is a model-based heuristic whose merits are tested empirically.
Assumptions & free parameters
free parameters (3)
- c (heteroskedasticity weights) =
data-dependent, n+1-dimensional
- theta (correlation lengthscales) =
data-dependent
- nu^2 and mu =
estimated by profile MLE
assumptions (5)
- ad hoc to paper The response obeys y(x) = mu + tau(x)Z(x) with stationary Z and tau(x) = nu/(c0 + r(x)'c).
- domain assumption Gaussian correlation function with diagonal lengthscales.
- ad hoc to paper Constraints c'c <= 1 and c >= 0 are sufficient for identifiability and stability.
- ad hoc to paper theta fixed at the rational-kriging estimate is adequate when optimizing c.
- domain assumption Candidate set of size 100(p+1)^2 represents the space well.
Cite this review
Pith. "Pith review of Active Learning via Heteroskedastic Rational Kriging." pith.science (2026). https://pith.science/paper/TDP4YYST
@misc{pith2026250710952,
author = {Pith},
title = {Pith review of: Active Learning via Heteroskedastic Rational Kriging},
year = {2026},
howpublished = {\url{https://pith.science/paper/TDP4YYST}},
note = {Machine review of arXiv:2507.10952}
}
read the original abstract
Active learning methods for emulating complex computer models that rely on stationary Gaussian processes tend to produce design points that uniformly fill the entire experimental region, which can be wasteful for functions which vary only in small regions. In this article, we propose a new Gaussian process model that captures the heteroskedasticity of the function. Active learning using this new model can place design points in the more interesting regions of the response surface, and thus obtain surrogate models with better accuracy. The proposed active learning method is compared with the state-of-the-art methods using simulations and two real datasets. It is found to have comparable or better performance relative to other non-stationary Gaussian process-based methods, but faster by orders of magnitude.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := #2 '...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in " " * FUNCTION format....
-
[3]
Ba, S. and Joseph, V. R. (2012), Composite Gaussian process models for emulating expensive functions, The Annals of Applied Statistics, 1838--1860
work page 2012
-
[4]
Banerjee, S., Carlin, B. P., and Gelfand, A. E. (2003), Hierarchical modeling and analysis for spatial data, New York, NY: Chapman and Hall/CRC
work page 2003
-
[5]
Ben-Ari, E. N. and Steinberg, D. M. (2007), Modeling data from computer experiments: an empirical comparison of kriging with MARS and projection pursuit regression, Quality Engineering, 19, 327--338
work page 2007
-
[6]
Booth, A. S. (2024), deepgp: Bayesian Deep Gaussian Processes using MCMC, R package version 1.1.2. available at https://CRAN.R-project.org/package=deepgp
work page 2024
-
[7]
Bornn, L., Shaddick, G., and Zidek, J. V. (2012), Modeling nonstationary processes through dimension expansion, Journal of the American Statistical Association, 107, 281--289
work page 2012
-
[8]
D., De Marchi, S., and Perracchione, E
Buhmann, M. D., De Marchi, S., and Perracchione, E. (2020), Analysis of a new class of rational RBF expansions, IMA Journal of Numerical Analysis, 40, 1972--1993
work page 2020
Show all 52 references
-
[9]
and Lawrence, N
Damianou, A. and Lawrence, N. D. (2013), Deep Gaussian processes, in Artificial intelligence and statistics, PMLR, pp. 207--215
2013
-
[10]
and Pepelyshev, A
Dette, H. and Pepelyshev, A. (2010), Generalized Latin hypercube design for computer experiments, Technometrics, 52, 421--429
2010
-
[11]
M., Girolami, M
Dunlop, M. M., Girolami, M. A., Stuart, A. M., and Teckentrup, A. L. (2018), How deep are deep Gaussian processes? Journal of Machine Learning Research, 19, 1--46
2018
-
[12]
(2007), Investigation of reliability method formulations in DAKOTA/UQ, Structure and Infrastructure Engineering, 3, 199--213
Eldred, M., Agarwal, H., Perez, V., Wojtkiewicz Jr, S., and Renaud, J. (2007), Investigation of reliability method formulations in DAKOTA/UQ, Structure and Infrastructure Engineering, 3, 199--213
2007
-
[13]
and Raftery, A
Gneiting, T. and Raftery, A. E. (2007), Strictly proper scoring rules, prediction, and estimation, Journal of the American statistical Association, 102, 359--378
2007
-
[14]
Gramacy, R. B. (2020), Surrogates: G aussian Process Modeling, Design and \ Optimization for the Applied Sciences , Boca Raton, FL: Chapman Hall/CRC
2020
-
[15]
Gramacy, R. B. and Apley, D. W. (2015), Local Gaussian process approximation for large computer experiments, Journal of Computational and Graphical Statistics, 24, 561--578
2015
-
[16]
Gramacy, R. B. and Lee, H. K. (2009), Adaptive design and analysis of supercomputer experiments, Technometrics, 51, 130--145
2009
-
[17]
Gramacy, R. B. and Lee, H. K. H. (2008), Bayesian treed Gaussian process models with an application to computer modeling, Journal of the American Statistical Association, 103, 1119--1130
2008
-
[18]
o m, H., Rousu, J., Kaski, S., and L \
Heinonen, M., Mannerstr \"o m, H., Rousu, J., Kaski, S., and L \"a hdesm \"a ki, H. (2016), Non-stationary gaussian process regression with hamiltonian monte carlo, in Artificial Intelligence and Statistics, PMLR, pp. 732--740
2016
-
[19]
and Kohn, W
Hohenberg, P. and Kohn, W. (1964), Inhomogeneous electron gas, Physical review, 136, B864
1964
-
[20]
and Joseph, V
Huang, C. and Joseph, V. R. (2024), rkriging: Kriging Modeling, R package version 1.0. available at https://CRAN.R-project.org/package=rkriging
2024
-
[21]
K., Losego, M
Huang, C., Ren, Y., McGuinness, E. K., Losego, M. D., Lively, R. P., and Joseph, V. R. (2021), Bayesian optimization of functional output in inverse problems, Optimization and Engineering, 22, 2553--2574
2021
-
[22]
Huang, W., Wang, K., Jay Breidt, F., and Davis, R. A. (2011), A class of stochastic volatility models for environmental applications, Journal of Time Series Analysis, 32, 364--377
2011
-
[23]
(2009), Rational radial basis function interpolation with applications to antenna design, Journal of computational and applied mathematics, 233, 889--904
Jakobsson, S., Andersson, B., and Edelvik, F. (2009), Rational radial basis function interpolation with applications to antenna design, Journal of computational and applied mathematics, 233, 889--904
2009
-
[24]
E., Moore, L
Johnson, M. E., Moore, L. M., and Ylvisaker, D. (1990), Minimax and Maximin Distance Designs, Journal of Statistical Planning and Inference, 26, 131--148
1990
-
[25]
Johnson, S. G. (2008), The NLopt nonlinear-optimization package, available at https://github.com/stevengj/nlopt
2008
-
[26]
Joseph, V. R. (2006), Limit kriging, Technometrics, 48, 458--466
2006
-
[27]
--- (2016), Space-filling designs for computer experiments: A review (with discussions), Quality Engineering, 28, 28--44
2016
-
[28]
--- (2024), Rational Kriging, Journal of the American Statistical Association, available at https://doi.org/10.1080/01621459.2024.2356296
2024
-
[29]
R., Gul, E., and Ba, S
Joseph, V. R., Gul, E., and Ba, S. (2015), Maximum projection designs for computer experiments, Biometrika, 102, 371--380
2015
-
[30]
and Joseph, V
Kang, L. and Joseph, V. R. (2016), Kernel approximation: From regression to interpolation, SIAM/ASA Journal on Uncertainty Quantification, 4, 112--129
2016
-
[31]
and Zacks, S
Kenett, R. and Zacks, S. (1998), Modern industrial statistics: design and control of quality and reliability, Pacific Grove, CA: Duxbury Press
1998
-
[32]
Krishna, A., Tran, H., Huang, C., Ramprasad, R., and Joseph, V. R. (2024), Adaptive exploration and optimization of materials crystal structures, INFORMS Journal on Data Science, 3, 68--83
2024
-
[33]
(2006), RDKit: Open-source cheminformatics
Landrum, G. (2006), RDKit: Open-source cheminformatics. 2006, a vailable at https://rdkit.org/
2006
-
[34]
D., Kong, L., Lundqvist, B
Lee, K., Murray, \'E . D., Kong, L., Lundqvist, B. I., and Langreth, D. C. (2010), Higher-accuracy van der Waals density functional, Physical Review B—Condensed Matter and Materials Physics, 82, 081101
2010
-
[35]
Leng, C. Z. and Losego, M. D. (2017), Vapor phase infiltration (VPI) for transforming polymers into organic--inorganic hybrid materials: a critical review of current progress and future challenges, Materials Horizons, 4, 747--771
2017
-
[36]
G., Joseph, V
Liu, Y., Tran, H., Huang, C., del Rio, B. G., Joseph, V. R., Losego, M., and Ramprasad, R. (2024), Accelerated predictions of the sublimation enthalpy of organic materials with machine learning, Materials Genome Engineering Advances
2024
-
[37]
MacKay, D. J. (1992), Information-based objective functions for active data selection, Neural computation, 4, 590--604
1992
-
[38]
R., Yang, V., and Wu, C
Mak, S., Sung, C.-L., Wang, X., Yeh, S.-T., Chang, Y.-H., Joseph, V. R., Yang, V., and Wu, C. F. J. (2018), An efficient surrogate model for emulation and physics extraction of large eddy simulations, Journal of the American Statistical Association, 113, 1443--1456
2018
-
[39]
and Tokdar, S
Montagna, S. and Tokdar, S. T. (2016), Computer emulation with nonstationary Gaussian processes, SIAM/ASA Journal on Uncertainty Quantification, 4, 26--47
2016
-
[40]
D., Mitchell, T
Morris, M. D., Mitchell, T. J., and Ylvisaker, D. (1993), Bayesian design and analysis of computer experiments: use of derivatives in surface prediction, Technometrics, 35, 243--255
1993
-
[41]
(2003), Automated CFD parameter studies on distributed parallel computers, in 16th AIAA Computational Fluid Dynamics Conference, p
Rogers, S., Aftosmis, M., Pandya, S., Chaderjian, N., Tejnil, E., and Ahmad, J. (2003), Automated CFD parameter studies on distributed parallel computers, in 16th AIAA Computational Fluid Dynamics Conference, p. 4229
2003
-
[42]
Sampson, P. D. and Guttorp, P. (1992), Nonparametric estimation of nonstationary spatial covariance structure, Journal of the American Statistical Association, 87, 108--119
1992
-
[43]
J., Williams, B
Santner, T. J., Williams, B. J., and Notz, W. I. (2019), The Design and Analysis of Computer Experiments, New York, NY: Springer
2019
-
[44]
Sarra, S. A. and Bai, Y. (2018), A rational radial basis function method for accurately resolving discontinuities and steep gradients, Applied Numerical Mathematics, 130, 131--142
2018
-
[45]
Sauer, A., Cooper, A., and Gramacy, R. B. (2023 a ), Non-stationary Gaussian process surrogates, arXiv preprint arXiv:2305.19242
2023 arXiv
-
[46]
B., and Higdon, D
Sauer, A., Gramacy, R. B., and Higdon, D. (2023 b ), Active learning for deep Gaussian process surrogates, Technometrics, 65, 4--18
2023
-
[47]
Schmidt, A. M. and O'Hagan, A. (2003), Bayesian inference for non-stationary spatial covariance structure via spatial deformations, Journal of the Royal Statistical Society Series B: Statistical Methodology, 65, 743--758
2003
-
[48]
and Bingham, D
Surjanovic, S. and Bingham, D. (2013), Virtual Library of Simulation Experiments: Test Functions and Datasets, available at https://www.sfu.ca/ ssurjano/about.html
2013
-
[49]
(2002), A class of globally convergent optimization methods based on conservative convex separable approximations, SIAM journal on optimization, 12, 555--573
Svanberg, K. (2002), A class of globally convergent optimization methods based on conservative convex separable approximations, SIAM journal on optimization, 12, 555--573
2002
-
[50]
Tolvanen, V., Jylänki, P., and Vehtari, A. (2014), Expectation propagation for nonstationary heteroscedastic Gaussian process regression, in 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP), pp. 1--6
2014
-
[51]
and Joseph, V
Vakayil, A. and Joseph, V. R. (2022), Data twinning, Statistical Analysis and Data Mining: The ASA Data Science Journal, 15, 598--610
2022
-
[52]
W., Booth, A
Wycoff, N., Smith, J. W., Booth, A. S., and Gramacy, R. B. (2024), Voronoi Candidates for Bayesian Optimization, arXiv preprint arXiv:2402.04922
2024 arXiv
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.