REVIEW 3 major objections 3 minor 1 cited by
Sobol' Matrices For Multi-Output Models With Quantified Uncertainty
T0 review · 3 major / 3 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper proposes a Sobol' matrix that measures how strongly each set of inputs drives the correlations between multiple outputs, with the classical Sobol' indices on the diagonal.
desk verdict Useful extension of Sobol' indices to multi-output correlations, but the central expected-value identity is false and needs correction before the paper can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the closed Sobol' matrix $S_m := V_m[y_m]/(D_M[y_M] \otimes D_M[y_M])$, where $y_m$ is the reduced model obtained by marginalizing inputs outside $m$, and the denominator is the outer product of output standard deviations of the full model. The argument is carried by representing the model as a multi-output model with quantified uncertainty, i.e. a stochastic process on $[0,1]^{M+1}$ with one undetermined uniform input appended to the determined inputs; marginalization and variance computation are then both expectations, so Fubini's theorem lets the paper evaluate them in the order that produces moments of the process. The first moment gives the Sobol' matrix itself, and the covariance of variances (the fourth-order tensor $W_{mm'}$, Equation (24)) feeds a Taylor-series error propagation formula that yields the standard error $T_m$. The benchmark compares these moment-based matrices against analytically computed matrices for test functions, using a Gaussian-process surrogate as the source of moments.
What would settle it
Take a model on $[0,1]^{M+1}$ with a heavy-tailed undetermined input, for example $y(u)=u_0+u_{\bar M}^{-1/4}$, which has finite mean but infinite fourth moment, and compare the Taylor standard error of Equation (26) against a large Monte Carlo estimate of the true standard deviation of the Sobol' matrix entry; the formula should fail to track the simulation, showing that the moment-existence assumption is doing real work.
Extended reading notes
Core claim
The central claim is that the tensor extension of the scalar Sobol' index is the closed Sobol' matrix of Equation (14), defined through the coefficient-of-determination form rather than by elementwise division of covariance matrices, which can hit zero denominators. Each entry lies in $[-1,1]$; the diagonal reproduces the classical Sobol' index for each output, and an off-diagonal entry $[S_m]_{l,l'}$ expresses the fraction of covariance between outputs $l$ and $l'$ that input axes $m$ account for, with negative values when those inputs push the two outputs in opposite directions. Existence and uniqueness are argued by embedding models as multi-output models with quantified uncertainty, and the matrix plus its standard error are reduced to the first two moment tensors of that stochastic process using Fubini's theorem and the Kolmogorov extension theorem, with the standard error obtained by a Taylor expansion of the covariance of marginal variances.
Load-bearing premise
The load-bearing premise is that the model's random outputs are well-behaved enough that the average of their fourth powers is finite, a premise the paper justifies with the claim that integrability on a finite space implies all powers are integrable--which is backwards.
Editorial extensions
If this is right
- The diagonal of every closed Sobol' matrix recovers the classical scalar Sobol' indices, so the matrix is a direct generalization rather than a separate sensitivity measure.
- The sign and magnitude of off-diagonal entries let an analyst tell which inputs strengthen a correlation between outputs and which inputs oppose it, pointing to which inputs to vary in order to change a trade-off.
- Because noise with constant variance cancels from the moment formula, Sobol' matrices and their standard errors are largely unaffected by additive output noise; the benchmarks still degrade when noise is so strong that the Gaussian-process surrogate itself stops predicting well.
- Total Sobol' matrices $S^T_{M-m}=S_M-S_m$ give a dimension-reduction criterion: input axes whose total matrix is near zero can be dropped, and the benchmark's total-index accuracy is nearly independent of $m$ up to seven input dimensions.
- The moment-based standard errors act as a diagnostic of numerical stability, with benchmark standardized scores mostly below 3 at the median and below 6 at the 90th percentile.
Reading between the lines
- Beyond the paper, the same two-moment reduction could be applied to any surrogate that supplies moment tensors, not only Gaussian processes; this would let the method scale to larger output counts by choosing a surrogate with cheap moment arithmetic.
- The signed off-diagonal structure suggests a natural extension the authors do not pursue: for each input axis, the Sobol' matrix can be read as a signed network on outputs, allowing sensitivity analysis to double as a screening tool for which inputs reconfigure the correlation structure.
- The noise-invariance result is proven for homoskedastic noise, and the benchmarks only use Gaussian noise; whether the cancellation survives heteroskedastic or non-Gaussian uncertainty is an open question that a direct test would settle.
- Adding an explicit finite-fourth-moment assumption would make the standard-error formulas unconditional; without it, the reported error bars rest on the reversed inclusion argument flagged above.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a generalization of scalar Sobol' indices to vector-valued outputs, called Sobol' matrices. For a multi-output model with quantified uncertainty (MQU), the closed Sobol' matrix is defined in Eq. (14) as the random variable S_m := V_m[y_m] / (D_M[y_M] ⊗ D_M[y_M]), whose diagonal entries are the classical Sobol' indices and whose off-diagonal entries measure input influence on output correlations. The paper derives moment-based formulas for the expected Sobol' matrices and their standard errors, expresses these through the first two moments of an MQU, implements them for multi-output Gaussian processes, and benchmarks the implementation against analytically computed Sobol' matrices of a 9-output test function with added noise.
Significance. If the moment formulas were correct, Sobol' matrices would be a natural and practically valuable extension of Sobol' indices: the diagonal is exactly the classical index, while the off-diagonal entries quantify how inputs affect correlations between outputs. The paper also provides a large benchmark campaign and explicit closed-form tensor expressions, which are useful engineering contributions. However, the central identity connecting the expectation of the ratio in Eq. (14) to a ratio of expectations in Eq. (21a) is false in general, and the moment-existence argument in Section 2 is invalid. These issues are load-bearing for the paper's main claims, so the current version does not establish the asserted statistics.
major comments (3)
- [Section 5, Eqs. (20b) and (21a)] The implication [D_m]_l = sqrt([V_m]_{l×l}) is not a consequence of the definition [D_m]_l = E_M[D_m[y_m]]_l. Jensen's inequality gives E[sqrt(X)] ≤ sqrt(E[X]), with equality only when X is constant. Therefore Eq. (21a) is not the expectation of the Sobol' matrix defined in Eq. (14); it is instead a ratio of expectations. This is not a cosmetic distinction: for any MQU for which D_M[y_M] varies with the undetermined input, E[V_m[y_m]/(D_M[y_M] ⊗ D_M[y_M])] differs from [V_m]/([D_M] ⊗ [D_M]). Because the Section 8 benchmarks compare moment-based estimates against MNU ground truth, whose denominator D_M[y_M] is deterministic, the discrepancy is invisible in the reported validation. This issue is independent of the moment-existence problem.
- [Section 2, paragraph following Definition 2] The assertion that Lebesgue integrability of y on a finite measure implies integrability of y^n for all n is backwards. On a probability space the inclusion is L^p ⊂ L^q when p ≥ q, so L^1 does not imply L^2 or L^4. Consequently Definition 2 does not guarantee the existence of the expected marginal variances [V_m] in Eq. (20a), let alone the covariance-of-variances tensor W_mm' in Eq. (24) or the error formulas in Eqs. (25)–(26). The authors should either strengthen Definition 2 to explicitly assume finite moments of sufficiently high order, or supply a different proof; as written, the standard-error formulas are not mathematically justified.
- [Section 5, Eqs. (25)–(26)] The Taylor-series standard error is an approximation, and the stated validity condition, that V_M[y_M] is well approximated by its mean, is not quantified. The benchmarks in Section 8 report standardized scores A/T, but they do not validate T against the true standard deviation of S_m for a case where D_M[y_M] varies with the undetermined input. The claim that Eq. (23a) is the standard error of the Sobol' matrix is therefore not established for general MQUs. This is closely connected to the first major comment and should be addressed together with it.
minor comments (3)
- [References] Reference [30] is cited as 'In preparation, 2025' and is the source of the analytic GP expressions and implementation details used in Section 8; the manuscript is not self-contained for reproducing the benchmarks without this companion reference.
- [Throughout] The text contains typographical errors that should be corrected, including 'Sobol' Ma trices' in the title, 'MUL TI-OUTPUT', 'pharamaceutical', 'analagous', and 'initally'.
- [Section 7, Eq. (40)] The symbol E is used both for the expectation operator and for the noise magnitude in Eq. (40); although context usually disambiguates the two, the dual use is confusing and should be resolved.
Circularity Check
No significant circularity: the closed Sobol' matrix is derived from MQU moments and benchmarked against analytically computed test-function matrices.
full rationale
The paper's central derivation (Sections 3-6) defines Sobol' matrices as tensor random variables and then expresses their expected values and standard errors through MQU moments using Fubini's theorem and the law of iterated expectations. The target quantities (MNU Sobol' matrices) are computed analytically from test functions, while the GP surrogate is fit only to noisy outputs; the GP's moments are then inserted into closed-form formulas, so no prediction is statistically forced by a fitted parameter. The self-references to [30] and RomCom [3] supply implementation details, not the derivation, and no uniqueness theorem from the authors' prior work is invoked to exclude alternatives. Two genuine mathematical weaknesses appear in the paper, but neither is circularity: the claim that L^1 integrability implies integrability of all powers (Section 2) reverses the finite-measure inclusion L^p subset L^q for p > q, and Eq. (21a) equates E[S_m] with a ratio of expectations, which is false unless the denominator is constant; both are correctness and validity risks, not instances of the derivation reducing to its own inputs. The benchmark's use of MNUs with deterministic denominators does mask the Eq. (21a) discrepancy, but that is an evaluation limitation rather than a circular step.
Assumptions & free parameters
free parameters (1)
- GP RBF kernel hyperparameters and noise variance =
not reported
assumptions (3)
- domain assumption Inputs are mutually independent and mapped through the probability integral transform to U(0,1).
- domain assumption All MQU moments needed for the construction exist and expectations can be reordered by Fubini.
- ad hoc to paper The Taylor approximation of the ratio defining Sobol' matrices is accurate enough for the reported standard errors.
invented entities (1)
-
Sobol' matrix (as a new mathematical statistic)
Cite this review
Pith. "Pith review of Sobol' Matrices For Multi-Output Models With Quantified Uncertainty." pith.science (2026). https://pith.science/paper/WJZUFOXU
@misc{pith2026250104602,
author = {Pith},
title = {Pith review of: Sobol' Matrices For Multi-Output Models With Quantified Uncertainty},
year = {2026},
howpublished = {\url{https://pith.science/paper/WJZUFOXU}},
note = {Machine review of arXiv:2501.04602}
}
read the original abstract
Variance based global sensitivity analysis measures the relevance of inputs to a single output using Sobol' indices. This paper extends the definition in a natural way to multiple outputs, directly measuring the relevance of inputs to the linkages between outputs in a correlation-like matrix of indices. The usual Sobol' indices constitute the diagonal of this matrix. Existence, uniqueness and uncertainty quantification are established by developing the indices from a putative multi-output model with quantified uncertainty. Sobol' matrices and their standard errors are related to the moments of the multi-output model, to enable calculation. These are benchmarked numerically against test functions (with added noise) whose Sobol' matrices are calculated analytically.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
Conditional Mean Independence and Global Sensitivity Analysis using Nearest Neighbor Graphs
A nearest-neighbor graph estimator of the normalized conditional mean discrepancy is consistent, rate-optimal in low dimension, asymptotically normal under the null, and yields a fast test and screening procedure.
Reference graph
Works this paper leans on
-
[1]
NumPy, https://numpy.org, Last accessed on 2025-01-06
work page 2025
-
[2]
PyTorch, https://pytorch.org, Last accessed on 2025-01-06
work page 2025
-
[3]
RomCom, https://github.com/miltonra/RomCom, Last accessed on 2025-01-06
work page 2025
-
[4]
SALib, https://salib.readthedocs.io/en/latest/, Last accessed on 2025-01-06
work page 2025
-
[5]
TensorFlow, https://tensorflow.org, Last accessed on 2025-01-06
work page 2025
-
[6]
Naomi Altman and Martin Krzywinski, The curse(s) of dimensionality , Nature Methods 15 (2018), no. 6, 399–400
work page 2018
-
[7]
Angus, The probability integral transform and related results , SIAM Review 36 (1994), no
John E. Angus, The probability integral transform and related results , SIAM Review 36 (1994), no. 4, 652–654
work page 1994
-
[8]
Mickael Binois and Nathan Wycoff, A survey on high-dimensional gaussian process modeling with application to bayesian optimization, November 2021
work page 2021
Show all 56 references
-
[9]
Blitzstein and Jessica Hwang, Introduction to probability, second edition, Taylor & Francis Group, 2019
Joseph K. Blitzstein and Jessica Hwang, Introduction to probability, second edition, Taylor & Francis Group, 2019
2019
-
[10]
McKay, and Brian J
Katherine Campbell, Michael D. McKay, and Brian J. Williams, Sensitivity analysis when model outputs are functions , Reliability Engineering & System Safety 91 (2006), no. 10-11, 1468–1472
2006
-
[11]
Ga¨ elle Chastaing, Fabrice Gamboa, and Cl´ ementine Prieur,Generalized hoeffding-sobol decomposition for dependent variables -application to sensitivity analysis , (2011)
2011
-
[12]
Wei Chen, Ruichen Jin, and Agus Sudjianto, Analytical variance-based global sensitivity analysis in simulation-based design under uncertainty , Journal of Mechanical Design 127 (2005), no. 5, 875
2005
-
[13]
6, 2177–2187
Kai Cheng, Zhenzhou Lu, and Kaichao Zhang, Multivariate output global sensitivity analysis using multi-output support vector regression, Structural and Multidisciplinary Optimization 59 (2019), no. 6, 2177–2187
2019
-
[14]
1, 897–921
Mathilde Chevreuil, R´ egis Lebrun, Anthony Nouy, and Prashant Rai, A least-squares method for sparse low rank approximation of multivariate functions , SIAM/ASA Journal on Uncertainty Quantification 3 (2015), no. 1, 897–921
2015
-
[15]
Davide Chicco, Matthijs J. Warrens, and Giuseppe Jurman, The coefficient of determination r-squared is more infor- mative than SMAPE, MAE, MAPE, MSE and RMSE in regression analysis evaluation , PeerJ Computer Science 7 (2021-07), no. 1, e623
2021
-
[16]
3, 273–297
Corinna Cortes and Vladimir Vapnik, Support-vector networks, Machine Learning 20 (1995), no. 3, 273–297
1995
-
[17]
7-8, 307–310
Fabrice Gamboa, Alexandre Janon, Thierry Klein, and Agn` es Lagnoux, Sensitivity indices for multivariate outputs , Comptes Rendus Mathematique 351 (2013), no. 7-8, 307–310
2013
-
[18]
6, 25–36
Oscar Garcia-Cabrejo and Albert Valocchi, Global sensitivity analysis for multivariate output using polynomial chaos expansion, Reliability Engineering & System Safety 126 (2014), no. 6, 25–36
2014
-
[19]
R. G. Ghanem and P. D. Spanos, Spectral techniques for stochastic finite elements, Archives of Computational Methods in Engineering 4 (1997), no. 1, 63–100
1997
-
[20]
6, Springer, 2017
Roger Ghanem, David Higdon, Houman Owhadi, et al., Handbook of uncertainty quantification, vol. 6, Springer, 2017
2017
-
[21]
Toshimitsu Homma and Andrea Saltelli, Importance measures in global sensitivity analysis of nonlinear models , Reli- ability Engineering & System Safety 52 (1996), no. 1, 1–17
1996
-
[22]
Ishigami and T
T. Ishigami and T. Homma, An importance quantification technique in uncertainty analysis for computer models ,
-
[23]
Kendall, Kendall’s advanced theory of statistics , John Wiley & Sons, 1994
Maurice G. Kendall, Kendall’s advanced theory of statistics , John Wiley & Sons, 1994
1994
-
[24]
Davar Khoshnevisan, Multiparameter processes, Springer Monographs in Mathematics Ser., Springer New York, New York, NY, 2002, Description based on publisher supplied metadata and other sources
2002
-
[25]
Katerina Konakli and Bruno Sudret, Global sensitivity analysis using low-rank tensor approximations , Reliability Engineering & System Safety 156 (2016), 64–83
2016
-
[26]
4, 450–459
Matieyendou Lamboni, Herv´ e Monod, and David Makowski, Multivariate sensitivity analysis to measure global con- tribution of input factors in dynamic models , Reliability Engineering & System Safety 96 (2011), no. 4, 450–459
2011
-
[27]
Benjamin Lamoureux, Nazih Mechbal, and Jean R´ emi Mass´ e,A combined sensitivity analysis and kriging surrogate modeling for early validation of health indicators , Reliability Engineering and System Safety 130 (2014), 12–26
2014
-
[28]
Qiming Liu, Nichen Tong, and Xu Han, A global sensitivity analysis method for multi-input multi-output system and its application in structural design , International Journal of Computational Methods 19 (2021), no. 06. SOBOL’ MATRICES FOR MULTI-OUTPUT MODELS WITH QUANTIFIED UN...
2021
-
[29]
3, 742–751
Amandine Marrel, Bertrand Iooss, B´ eatrice Laurent, and Olivier Roustant,Calculations of sobol indices for the gauss- ian process metamodel, Reliability Engineering & System Safety 94 (2009), no. 3, 742–751
2009
-
[30]
Milton and Solomon F
Robert A. Milton and Solomon F. Brown, In preparation, 2025
2025
-
[31]
Jeremy E. Oakley and Anthony O’Hagan, Probabilistic sensitivity analysis of complex models: a bayesian approach , Journal of the Royal Statistical Society: Series B (Statistical Methodology) 66 (2004), no. 3, 751–769
2004
-
[32]
Carl Edward Rasmussen and Christopher K. I. Williams, Gaussian processes for machine learning (adaptive compu- tation and machine learning series) , The MIT Press, 2005
2005
-
[33]
Saman Razavi, Anthony Jakeman, Andrea Saltelli, Cl´ ementine Prieur, Bertrand Iooss, Emanuele Borgonovo, Elmar Plischke, Samuele Lo Piano, Takuya Iwanaga, William Becker, Stefano Tarantola, Joseph H.A. Guillaume, John Jakeman, Hoshin Gupta, Nicola Melillo, Giovanni Rabitti, Vi...
2021
-
[34]
L. C. G. Rogers and D. Williams, Diffusions, markov processes, and martingales , Cambridge University Press, 2000
2000
-
[35]
Welch, Toby J
Jerome Sacks, William J. Welch, Toby J. Mitchell, and Henry P. Wynn, Design and analysis of computer experiments , Statistical Science 4 (1989), no. 4, 409–423
1989
-
[36]
Andrea Saltelli, Ksenia Aleksankina, William Becker, Pamela Fennell, Federico Ferretti, Niels Holst, Sushan Li, and Qiongli Wu, Why so many published sensitivity analyses are false: A systematic review of sensitivity analysis practices , Environmental Modelling & Software 114 ...
2019
-
[37]
design and estimator for the total sensitivity index , 181 (2010-02), no
Andrea Saltelli, Paola Annoni, Ivano Azzini, Francesca Campolongo, Marco Ratto, and Stefano Tarantola, Variance based sensitivity analysis of model output. design and estimator for the total sensitivity index , 181 (2010-02), no. 2, 259–270
2010
-
[38]
Andrea Saltelli, Marco Ratto, Terry Andres, Francesca Campolongo, Jessica Cariboni, Debora Gatelli, Michaela Saisana, and Stefano Tarantola, Global sensitivity analysis , Wiley-Interscience, 2008
2008
-
[39]
Eric Schechter, Handbook of analysis and its foundations , Academic Press, San Diego, 1997
1997
-
[40]
Valeriy Skorokhod, Basic principles and applications of probability theory , Springer Berlin Heidelberg, 2005
2005
-
[41]
I Sobol, Sensitivity estimates for nonlinear mathematical models , Mathematical Modelling and Computational Exper- iments 4 (1993), 407–414
1993
-
[42]
I. M. Sobol, Global sensitivity indices for nonlinear mathematical models and their monte carlo estimates, Mathematics and Computers in Simulation 55 (2001), 271–280
2001
-
[43]
I. M. Sobol and S. S. Kucherenko, Global sensitivity indices for nonlinear mathematical models. review , Wilmott 2005 (2005), no. 1, 56–61
2005
-
[44]
Ankur Srivastava, Arun K. Subramaniyan, and Liping Wang, Analytical global sensitivity analysis with gaussian processes, Artificial Intelligence for Engineering Design, Analysis and Manufacturing 31 (2017), no. 03, 235–250
2017
-
[45]
7, 964–979
Bruno Sudret, Global sensitivity analysis using polynomial chaos expansions , Reliability Engineering & System Safety 93 (2008), no. 7, 964–979
2008
-
[46]
Felipe A. C. Viana, A tutorial on latin hypercube design of experiments , Quality and Reliability Engineering Interna- tional 32 (2015), no. 5, 1975–1985
2015
-
[47]
Alfonso Villani, Another note on the inclusion L p(µ) ⊂ Lq(µ), The American Mathematical Monthly 92 (1985), no. 7, 485
1985
-
[48]
Philippe Wiederkehr, Global sensitivity analysis with dependent inputs , Master’s thesis, 2018
2018
-
[49]
David Williams, Probability with martingales , Cambridge University Press, 1991
1991
-
[50]
Zeping Wu, Donghui Wang, Patrick Okolo N, Fan Hu, and Weihua Zhang, Global sensitivity analysis using a gaussian radial basis function metamodel , Reliability Engineering & System Safety 154 (2016), 171–179
2016
-
[51]
Sinan Xiao, Zhenzhou Lu, and Feifei Qin, Estimation of the generalized sobol’s sensitivity index for multivariate output model using unscented transformation , Journal of Structural Engineering 143 (2017), no. 5
2017
-
[52]
Dongbin Xiu, Numerical methods for stochastic computations: A spectral method approach, Princeton University Press, 2010
2010
-
[53]
2, 619–644
Dongbin Xiu and George Em Karniadakis, The wiener–askey polynomial chaos for stochastic differential equations , SIAM Journal on Scientific Computing 24 (2002), no. 2, 619–644
2002
-
[54]
Can Xu, Ping Zhu, Zhao Liu, and Wei Tao, Mapping-based hierarchical sensitivity analysis for multilevel systems with multidimensional correlations, Journal of Mechanical Design 143 (2020), no. 1
2020
-
[55]
Kaichao Zhang, Zhenzhou Lu, Kai Cheng, Laijun Wang, and Yanling Guo, Global sensitivity analysis for multivariate output model and dynamic models , Reliability Engineering & System Safety 204 (2020), 107195
2020
-
[1990]
First International Symposium on Uncertainty Modeling and Analysis, IEEE Comput
Proceedings. First International Symposium on Uncertainty Modeling and Analysis, IEEE Comput. Soc. Press
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.