REVIEW 1 major objections 88 references
Reviewed by Pith at T0; open to challenge.
T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →
T0 review · grok-4.3
A new covariance function for mixed inputs paired with Vecchia approximation extends scalable emulation methods to computer experiments with both quantitative and qualitative factors.
2026-06-28 11:39 UTC pith:M2IEH7K6
load-bearing objection The paper gives a new additive covariance for mixed quantitative-qualitative inputs in GPs and pairs it with Vecchia, but the structural compatibility with Vecchia's local conditioning is not obviously guaranteed. the 1 major comments →
Emulators for Large-scale Computer Experiments with Quantitative and Qualitative Inputs
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that a covariance function integrating additive Gaussian processes to handle mixed quantitative and qualitative inputs, when used with the Vecchia approximation, forms a modeling framework under which methods already developed for large-scale computer experiments can be effectively extended while preserving the required accuracy and scalability.
What carries the argument
The new covariance function that integrates additive Gaussian processes with Vecchia approximation; it encodes the mixed-input structure while supplying the sparsity needed for large-scale computation.
Load-bearing premise
That the proposed covariance function for mixed inputs remains accurate and computationally tractable once the Vecchia approximation is applied.
What would settle it
Direct numerical comparison on a large mixed-input dataset in which the new emulator shows either substantially higher prediction error or no computational gain relative to standard large-scale methods applied only to the quantitative inputs.
If this is right
- Existing large-scale emulation algorithms developed for quantitative inputs become directly usable on problems that also contain qualitative inputs.
- Prediction and uncertainty quantification remain feasible at scales previously limited to purely quantitative experiments.
- The same modeling structure supports extension of multiple different large-scale techniques rather than requiring entirely new algorithms for the mixed case.
Where Pith is reading between the lines
- The framework might be tested by replacing the Vecchia step with other sparse approximations to check whether accuracy holds under different sparsity patterns.
- Applications in design optimization could treat qualitative factors as first-class inputs without separate categorical encoding tricks.
- The additive structure may allow straightforward incorporation of additional input types if they can be expressed through similar additive covariance terms.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a novel scalable framework for emulating large-scale computer experiments with mixed quantitative and qualitative inputs. The core contribution is a new covariance function that integrates additive Gaussian processes to accommodate the mixed inputs, paired with the Vecchia approximation to achieve computational scalability. The authors claim that this modeling framework enables effective extension of existing large-scale emulation methods to the mixed-input setting.
Significance. If the new additive covariance construction can be shown to preserve sufficient accuracy under Vecchia approximation, the work would address a practical challenge in computer experiment emulation and provide a reusable template for extending other scalable GP methods. The proposal of an additive GP covariance for mixed inputs combined with Vecchia is a direct response to a common modeling need, but its load-bearing assumption requires explicit support.
major comments (1)
- [Abstract / Proposed modeling framework and demonstration] The central scalability claim rests on the unverified assumption that the proposed additive covariance for mixed inputs can be paired with Vecchia approximation without material loss of accuracy. Vecchia relies on ordered conditional approximations that assume local dependence; an additive construction separating quantitative and qualitative kernels can induce dense cross-factor blocks. No proof that the resulting precision matrix admits a sufficiently sparse Cholesky factor, nor any empirical bound on the Kullback-Leibler divergence between exact and Vecchia likelihoods, is supplied to substantiate the claim that existing large-scale methods extend effectively.
Simulated Author's Rebuttal
We thank the referee for their constructive comments on our manuscript. The primary concern is the need for stronger verification that the proposed additive covariance preserves accuracy under the Vecchia approximation. We address this point below and indicate the revisions we will make.
read point-by-point responses
-
Referee: [Abstract / Proposed modeling framework and demonstration] The central scalability claim rests on the unverified assumption that the proposed additive covariance for mixed inputs can be paired with Vecchia approximation without material loss of accuracy. Vecchia relies on ordered conditional approximations that assume local dependence; an additive construction separating quantitative and qualitative kernels can induce dense cross-factor blocks. No proof that the resulting precision matrix admits a sufficiently sparse Cholesky factor, nor any empirical bound on the Kullback-Leibler divergence between exact and Vecchia likelihoods, is supplied to substantiate the claim that existing large-scale methods extend effectively.
Authors: We acknowledge that the current manuscript does not supply a formal proof of sparsity in the precision matrix or explicit empirical bounds on the KL divergence. The empirical demonstrations in the paper show that the overall framework performs well on large mixed-input problems, but they do not isolate the approximation error in the manner suggested. In the revision we will add a dedicated subsection that (i) derives the sparsity pattern induced by the additive covariance under standard Vecchia orderings and (ii) reports numerical KL-divergence comparisons across a range of quantitative/qualitative dimensions and sample sizes. These additions will directly address the concern that the additive construction may produce dense cross-factor blocks incompatible with Vecchia. revision: yes
Circularity Check
New covariance proposal paired with Vecchia; derivation self-contained with no reduction to inputs or self-citations
full rationale
The paper introduces a novel covariance function based on additive GPs for mixed inputs and combines it with Vecchia approximation. No equations or claims in the provided abstract reduce a result to a fitted parameter renamed as prediction, a self-definitional loop, or a load-bearing self-citation chain. The central contribution is the construction itself rather than a derived quantity forced by prior inputs. This matches the default case of a self-contained methodological proposal with independent content.
Axiom & Free-Parameter Ledger
invented entities (1)
-
new covariance function for additive GPs
no independent evidence
read the original abstract
Computer experiments with both quantitative and qualitative inputs have become common across various areas. However, constructing accurate and computationally efficient emulators for such experiments at large scales remains a significant challenge. We propose a novel, scalable framework for emulating computer experiments with mixed inputs. Our approach is based on a new covariance function integrating additive Gaussian Processes (GPs) to handle the mixed inputs, with Vecchia approximation for scalability. We demonstrate that methods for large-scale computer experiments can be effectively extended when paired with our proposed modeling framework.
Figures
Reference graph
Works this paper leans on
-
[1]
SIAM/ASA Journal on Uncertainty Quantification , volume=
Grouped orthogonal arrays for computer experiments with grouped inputs , author=. SIAM/ASA Journal on Uncertainty Quantification , volume=. 2025 , publisher=
2025
-
[2]
Statistics in Industry and Government , volume=
Active learning of computer experiment with both quantitative and qualitative inputs , author=. Statistics in Industry and Government , volume=. 2025 , publisher=
2025
-
[3]
2013 , howpublished =
2013
-
[4]
Fully bayesian inference for latent variable
Yerramilli, Suraj and Iyer, Akshay and Chen, Wei and Apley, Daniel W , journal=. Fully bayesian inference for latent variable. 2023 , publisher=
2023
-
[5]
Journal of Complexity , number=
Quasi-regression , author=. Journal of Complexity , number=. 2001 , pages=
2001
-
[6]
Composite
Ba, Shan and Joseph, V Roshan , journal=. Composite. 2012 , pages=
2012
-
[7]
McGraw-Hill , year=
Matrix methods for engineers and scientists , author=. McGraw-Hill , year=
-
[8]
43rd International Conference on Parallel Processing , volume=
Fast parallel algorithms for edge-switching to achieve a target visit rate in heterogeneous graphs , author=. 43rd International Conference on Parallel Processing , volume=
-
[9]
AIAA Journal , number=
Efficient Global Reliability Analysis for Non-Linear Implicit Performance Functions , author=. AIAA Journal , number=. 2008 , pages=
2008
-
[10]
2006 , publisher=
Pattern Recognition and Machine Learning , author=. 2006 , publisher=
2006
-
[11]
Adaptive-region sequential design with quantitative and qualitative factors in application to
Cai, Xia and Xu, Li and Lin, C Devon and Hong, Yili and Deng, Xinwei , journal=. Adaptive-region sequential design with quantitative and qualitative factors in application to. 2024 , pages=
2024
-
[12]
MOANA: Modeling and analyzing
Cameron, Kirk W and Anwar, Ali and Cheng, Yue and Xu, Li and Li, Bo and Ananth, Uday and Bernard, Jon and Jearls, Chandler and Lux, Thomas and Hong, Yili and others , journal=. MOANA: Modeling and analyzing. 2019 , pages=
2019
-
[13]
2024 , note =
lhs: Latin Hypercube Samples , author =. 2024 , note =
2024
-
[14]
Journal of Quality Technology , number=
Entropy-based adaptive design for contour finding and estimating reliability , author=. Journal of Quality Technology , number=. 2023 , pages=
2023
-
[15]
IEEE Transactions on Information Theory , volume=
Nearest neighbor pattern classification , author=. IEEE Transactions on Information Theory , volume=
-
[16]
Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume=
Fixed rank kriging for very large spatial data sets , author=. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume=. 2008 , pages=
2008
-
[17]
Additive
Deng, X and Lin, C Devon and Liu, K-W and Rowe, RK , journal=. Additive. 2017 , publisher=
2017
-
[18]
Labeling examples that matter: Relevance-based active learning with
Freytag, Alexander and Rodner, Erik and Bodesheim, Paul and Denzler, Joachim , journal=. Labeling examples that matter: Relevance-based active learning with
-
[19]
Journal of Computational and Graphical Statistics , volume=
Covariance tapering for interpolation of large spatial datasets , author=. Journal of Computational and Graphical Statistics , volume=
-
[20]
Journal of Multivariate Analysis , number=
Compactly supported correlation functions , author=. Journal of Multivariate Analysis , number=
-
[21]
2018 , organization =
Machine Learning Tools , author =. 2018 , organization =
2018
-
[22]
Biometrics , year=
A general coefficient of similarity and some of its properties , author=. Biometrics , year=
-
[23]
Gramacy, Robert B and Apley, Daniel W , journal=. Local. 2015 , pages=
2015
-
[24]
Gramacy, Robert B , journal=
-
[25]
2021 , pages=
Guinness, Joseph , journal=. 2021 , pages=
2021
-
[26]
The Bell system technical journal , volume=
Error detecting and error correcting codes , author=. The Bell system technical journal , volume=
-
[27]
Convex combination of
Harari, Ofir and Steinberg, David M , journal=. Convex combination of. 2014 , pages=
2014
-
[28]
Advances in Neural Information Processing Systems , volume=
Predictive entropy search for efficient global optimization of black-box functions , author=. Advances in Neural Information Processing Systems , volume=
-
[29]
Journal of Global Optimization , number=
Efficient global optimization of expensive black-box functions , author=. Journal of Global Optimization , number=. 1998 , pages=
1998
-
[30]
Quality Engineering , volume=
Space-filling designs for computer experiments: A review , author=. Quality Engineering , volume=
-
[31]
Technometrics , volume=
SPlit: An optimal method for data splitting , author=. Technometrics , volume=. 2022 , pages=
2022
-
[32]
A general framework for
Katzfuss, Matthias and Guinness, Joseph , journal=. A general framework for. 2021 , pages=
2021
-
[33]
2020 , pages=
Katzfuss, Matthias and Guinness, Joseph and Gong, Wenlong and Zilber, Daniel , journal=. 2020 , pages=
2020
-
[34]
Katzfuss, Matthias and Guinness, Joseph and Lawrence, Earl , journal=. Scaled. 2022 , pages=
2022
-
[35]
Journal of the American Statistical Association , volume=
Covariance tapering for likelihood-based estimation in large spatial data sets , author=. Journal of the American Statistical Association , volume=. 2008 , publisher=
2008
-
[36]
Journal of the Royal Statistical Society: Series B (Statistical Methodology) , number=
Bayesian calibration of computer models , author=. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , number=. 2001 , pages=
2001
-
[37]
Adaptive exploration-exploitation active learning of
Kontoudis, George P and Otte, Michael , journal=. Adaptive exploration-exploitation active learning of
-
[38]
Devon Lin and Xinwei Deng , year =
Jiayi Li and Qian Xiao and Abhyuday Mandal and C. Devon Lin and Xinwei Deng , year =. Easy-to-Interpret
-
[39]
Handbook of Design and Analysis of Experiments
Latin hypercubes and space-filling designs , author=. Handbook of Design and Analysis of Experiments. Editors Angela Dean, Max Morris, John Stufken, Derek Bingham , volume=
-
[40]
2023 , organization =
Cluster Analysis , author =. 2023 , organization =
2023
-
[41]
Technometrics , volume=
A comparison of three methods for selecting values of input variables in the analysis of output from a computer code , author=. Technometrics , volume=
-
[42]
Monte Carlo implementation of
Neal, Radford M , journal=. Monte Carlo implementation of
-
[43]
Probabilistic sensitivity analysis of complex models: a
Oakley, Jeremy E and O'Hagan, Anthony , journal=. Probabilistic sensitivity analysis of complex models: a
-
[44]
Statistics and computing , volume=
Unconstrained parametrizations for variance-covariance matrices , author=. Statistics and computing , volume=
-
[45]
2008 , pages=
Qian, Peter Z G and Wu, Huaiqing and Wu, CF Jeff , journal=. 2008 , pages=
2008
-
[46]
Technometrics , number=
Sequential experiment design for contour estimation from complex computer codes , author=. Technometrics , number=. 2008 , pages=
2008
-
[47]
DiceKriging, DiceOptim: Two
Roustant, Olivier and Ginsbourger, David and Deville, Yves , journal=. DiceKriging, DiceOptim: Two
-
[48]
Statistical Science , number=
Design and analysis of computer experiments , author=. Statistical Science , number=. 1989 , pages=
1989
-
[49]
International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation , volume=
High performance computing and industry 4.0: Experiences from the disrupt project , author=. International Conference on Embedded Computer Systems: Architectures, Modeling, and Simulation , volume=
-
[50]
2003 , publisher=
The design and analysis of computer experiments , author=. 2003 , publisher=
2003
-
[51]
SIAM/ASA Journal on Uncertainty Quantification , volume=
Adaptive Design for Contour Estimation from Computer Experiments with Quantitative and Qualitative Inputs , author=. SIAM/ASA Journal on Uncertainty Quantification , volume=. 2025 , publisher=
2025
-
[52]
Srinivas, Niranjan and Krause, Andreas and Kakade, Sham M and Seeger, Matthias , journal=
-
[53]
Journal of Statistical Planning and Inference , volume=
Energy statistics: A class of statistics based on distances , author=. Journal of Statistical Planning and Inference , volume=. 2013 , pages=
2013
-
[54]
2022 , organization =
Data Twinning , author =. 2022 , organization =
2022
-
[55]
Statistical Analysis and Data Mining: The ASA Data Science Journal , volume=
Data twinning , author=. Statistical Analysis and Data Mining: The ASA Data Science Journal , volume=
-
[56]
A Global-Local Approximation Framework for Large-Scale
Vakayil, Akhil and Joseph, V Roshan , journal=. A Global-Local Approximation Framework for Large-Scale
-
[57]
Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=
Estimation and model identification for continuous spatial processes , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 1988 , pages=
1988
-
[58]
Cambridge Monographs on Applied and Computational Mathematics , year=
Scattered data approximation , author=. Cambridge Monographs on Applied and Computational Mathematics , year=
-
[59]
2006 , publisher=
Williams, Christopher KI and Rasmussen, Carl Edward , volume=. 2006 , publisher=
2006
-
[60]
Construction of maximin distance
Xiao, Qian and Xu, Hongquan , journal=. Construction of maximin distance. 2017 , publisher=
2017
-
[61]
2021 , pages=
Xiao, Qian and Mandal, Abhyuday and Lin, C Devon and Deng, Xinwei , journal=. 2021 , pages=
2021
-
[62]
Journal of Statistical Theory and Practice , volume=
Global fitting of the response surface via estimating multiple contours of a simulator , author=. Journal of Statistical Theory and Practice , volume=. 2020 , publisher=
2020
-
[63]
Mixed-input
Zhang, Qiong and Chien, Peter and Liu, Qing and Xu, Li and Hong, Yili , journal=. Mixed-input
-
[64]
A latent variable approach to
Zhang, Yichi and Tao, Siyu and Chen, Wei and Apley, Daniel W , journal=. A latent variable approach to. 2020 , pages=
2020
-
[65]
Scientific Reports , volume=
Bayesian optimization for materials design with mixed quantitative and qualitative variables , author=. Scientific Reports , volume=
-
[66]
Quality Engineering , volume=
Computer experiments with qualitative and quantitative variables: A review and reexamination , author=. Quality Engineering , volume=
-
[67]
Technometrics , volume=
A simple approach to emulation for computer models with qualitative and quantitative factors , author=. Technometrics , volume=. 2011 , pages=
2011
-
[68]
Statistica Sinica , volume=
Doubly coupled designs for computer experiments with both qualitative and quantitative factors , author=. Statistica Sinica , volume=. 2023 , pages=
2023
-
[69]
Qian, Peter ZG , journal=. Sliced. 2012 , pages=
2012
-
[70]
arXiv preprint arXiv:2206.01409 , volume=
Hybrid models for mixed variables in bayesian optimization , author=. arXiv preprint arXiv:2206.01409 , volume=
-
[71]
Cross-Validation--based Adaptive Sampling for
Mohammadi, Hossein and Challenor, Peter and Williamson, Daniel and Goodfellow, Marc , journal=. Cross-Validation--based Adaptive Sampling for. 2022 , publisher=
2022
-
[72]
Category tree
Lin, Wei-Ann and Sung, Chih-Li and Chen, Ray-Bing , journal=. Category tree. 2024 , publisher=
2024
-
[73]
2008 , publisher=
Engineering design via surrogate modelling: a practical guide , author=. 2008 , publisher=
2008
-
[74]
Gramacy , publisher =
Robert B. Gramacy , publisher =. Surrogates:
-
[75]
2022 , organization =
foreach: Provides Foreach Looping Construct , author =. 2022 , organization =
2022
-
[76]
2022 , organization =
doParallel: Foreach Parallel Adaptor for the 'parallel' Package , author =. 2022 , organization =
2022
-
[77]
2024 , organization =
lhs: Latin Hypercube Samples , author =. 2024 , organization =
2024
-
[78]
Genetic Optimization Using Derivatives: The
Walter R. Genetic Optimization Using Derivatives: The. Journal of Statistical Software , year =
-
[79]
Vecchia-approximated deep
Sauer, Annie and Cooper, Andrew and Gramacy, Robert B , journal=. Vecchia-approximated deep
-
[80]
Towards Global Optimiation 2 , volume=
The global optimization problem: an introduction , author=. Towards Global Optimiation 2 , volume=. 1978 , publisher=
1978
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.