{"id":"d6218e45-927b-40b2-b793-3ac6c3a9a244","arxiv_id":"2501.04602","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"Sobol' indices are generalized to a full matrix whose off-diagonal entries measure how inputs influence correlations between output variables, computable from a multi-output model's first two moments.","lead":"This paper extends variance-based sensitivity analysis (Sobol' indices) from single outputs to multiple outputs, producing a whole matrix that shows how each input affects the correlation between outputs. It gives formulas for the matrix and its error using a model's mean and covariance, and tests them on noisy benchmark functions.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (21a) equates the expectation of a random ratio with a ratio of expectations; this is generally false and undermines the moment-based computation of the closed Sobol' matrix.","rationale":"The paper's central claim is that the closed Sobol' matrix of an MQU is computed from the first two moments and its standard error from the Taylor expansion of those moments. The linchpin of this claim is Eq. (21a), which converts the random-variable definition (14) into a deterministic moment formula. That conversion is mathematically invalid: it replaces E[A/B] with E[A]/E[B] and E[sqrt(X)] with sqrt(E[X]), equalities that hold only when the denominator is constant. This is not a matter of missing moment assumptions; it fails for the simplest stochastic model with finite moments of all orders. The reader's identified weakness (the false L^p inclusion in Section 2) is a real but secondary defect: it affects whether certain expectations are finite, whereas Eq. (21a) is wrong even when every expectation exists. The numerical benchmark cannot reveal the error because the ground-truth MNUs have zero denominator variance, making the two quantities coincide. A concrete two-input, one-output example settles the issue immediately. The correct verdict remains conditional rather than outright rejection, because the paper's central construction could be repaired by redefining the reported Sobol' matrix as a ratio of expected moments (with an explicit approximation or bias statement) or by deriving the true expectation via a delta method. But as it stands, the paper's claim that Eq. (21a) is the expectation of the random Sobol' matrix is not supported, and the standard-error machinery inherits the problem.","tokens_in":19556,"tokens_out":13690,"duration_ms":131500,"concrete_test":"Consider M=2, L=1, with y(u0,u1,u2) = u0*u2 + u1 and u0,u1,u2 iid U(0,1). For m={0}, y_m = E_{u1}[y] = u0*u2 + 1/2, so V_m = u2^2/12 and V_M = Var(u0*u2+u1) = (u2^2+1)/12, D_M = sqrt((u2^2+1)/12). The exact expectation of the random variable in Eq. (14) is E[S_m] = E[u2^2/(u2^2+1)] = 1 - π/4 ≈ 0.2146. Eq. (21a) yields [V_m] = 1/36, [V_M] = 1/9, and hence a quotient of 1/4 = 0.25. The mismatch shows the equality is false. If the authors instead intend Eq. (21a) as a definition of a different summary statistic, that must be stated explicitly and the standard-error formulas must be re-derived for that statistic.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Equation (14) defines the closed Sobol' matrix as a random variable S_m := V_m[y_m] / (D_M[y_M] ⊗ D_M[y_M]) depending on the undetermined input. Section 5 then states (Eq. 21a) that the expected Sobol' matrix E_M[S_m] equals [V_m]/([D_M]⊗[D_M]), where [V_m] := E_M V_m[y_m] and [D_M] := E_M D_M[y_M]; equivalently, Eq. (20b) asserts E[sqrt(X)] = sqrt(E[X]) for the diagonal. Both identities require the denominator D_M[y_M] to be constant. For a genuine multi-output model with quantified uncertainty, D_M[y_M] varies with the undetermined input. By Jensen's inequality, E[sqrt(X)] < sqrt(E[X]) unless X is constant, and E[A/B] ≠ E[A]/E[B] in general. Thus the moment-based quantity computed in Eq. (21a) is not the expectation of the Sobol' matrix defined in Eq. (14); it is a different ratio-of-expectations estimator. Because the benchmark in Section 8 compares against MNUs (whose denominator is deterministic), this discrepancy is invisible in the reported numerical validation. This issue is independent of the moment-existence concern in Section 2: even if all moments exist, the equality in Eq. (21a) is false.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a generalization of scalar Sobol' indices to vector-valued outputs, called Sobol' matrices. For a multi-output model with quantified uncertainty (MQU), the closed Sobol' matrix is defined in Eq. (14) as the random variable S_m := V_m[y_m] / (D_M[y_M] ⊗ D_M[y_M]), whose diagonal entries are the classical Sobol' indices and whose off-diagonal entries measure input influence on output correlations. The paper derives moment-based formulas for the expected Sobol' matrices and their standard errors, expresses these through the first two moments of an MQU, implements them for multi-output Gaussian processes, and benchmarks the implementation against analytically computed Sobol' matrices of a 9-output test function with added noise.","tokens_in":19884,"tokens_out":7746,"duration_ms":79463,"significance":"If the moment formulas were correct, Sobol' matrices would be a natural and practically valuable extension of Sobol' indices: the diagonal is exactly the classical index, while the off-diagonal entries quantify how inputs affect correlations between outputs. The paper also provides a large benchmark campaign and explicit closed-form tensor expressions, which are useful engineering contributions. However, the central identity connecting the expectation of the ratio in Eq. (14) to a ratio of expectations in Eq. (21a) is false in general, and the moment-existence argument in Section 2 is invalid. These issues are load-bearing for the paper's main claims, so the current version does not establish the asserted statistics.","major_comments":[{"comment":"The implication [D_m]_l = sqrt([V_m]_{l×l}) is not a consequence of the definition [D_m]_l = E_M[D_m[y_m]]_l. Jensen's inequality gives E[sqrt(X)] ≤ sqrt(E[X]), with equality only when X is constant. Therefore Eq. (21a) is not the expectation of the Sobol' matrix defined in Eq. (14); it is instead a ratio of expectations. This is not a cosmetic distinction: for any MQU for which D_M[y_M] varies with the undetermined input, E[V_m[y_m]/(D_M[y_M] ⊗ D_M[y_M])] differs from [V_m]/([D_M] ⊗ [D_M]). Because the Section 8 benchmarks compare moment-based estimates against MNU ground truth, whose denominator D_M[y_M] is deterministic, the discrepancy is invisible in the reported validation. This issue is independent of the moment-existence problem.","section":"Section 5, Eqs. (20b) and (21a)"},{"comment":"The assertion that Lebesgue integrability of y on a finite measure implies integrability of y^n for all n is backwards. On a probability space the inclusion is L^p ⊂ L^q when p ≥ q, so L^1 does not imply L^2 or L^4. Consequently Definition 2 does not guarantee the existence of the expected marginal variances [V_m] in Eq. (20a), let alone the covariance-of-variances tensor W_mm' in Eq. (24) or the error formulas in Eqs. (25)–(26). The authors should either strengthen Definition 2 to explicitly assume finite moments of sufficiently high order, or supply a different proof; as written, the standard-error formulas are not mathematically justified.","section":"Section 2, paragraph following Definition 2"},{"comment":"The Taylor-series standard error is an approximation, and the stated validity condition, that V_M[y_M] is well approximated by its mean, is not quantified. The benchmarks in Section 8 report standardized scores A/T, but they do not validate T against the true standard deviation of S_m for a case where D_M[y_M] varies with the undetermined input. The claim that Eq. (23a) is the standard error of the Sobol' matrix is therefore not established for general MQUs. This is closely connected to the first major comment and should be addressed together with it.","section":"Section 5, Eqs. (25)–(26)"}],"minor_comments":[{"comment":"Reference [30] is cited as 'In preparation, 2025' and is the source of the analytic GP expressions and implementation details used in Section 8; the manuscript is not self-contained for reproducing the benchmarks without this companion reference.","section":"References"},{"comment":"The text contains typographical errors that should be corrected, including 'Sobol' Ma trices' in the title, 'MUL TI-OUTPUT', 'pharamaceutical', 'analagous', and 'initally'.","section":"Throughout"},{"comment":"The symbol E is used both for the expectation operator and for the noise magnitude in Eq. (40); although context usually disambiguates the two, the dual use is confusing and should be resolved.","section":"Section 7, Eq. (40)"}],"recommendation":"major_revision","confidential_remarks":"The paper is original and likely worth a major revision rather than rejection. The most serious problem is that Eq. (20b) conflates E[sqrt(X)] with sqrt(E[X]), and Eq. (21a) therefore computes a different object from the expectation of the Sobol' matrix defined in Eq. (14); the deterministic MNU benchmark masks this discrepancy. The moment-existence issue is easy to repair by strengthening Definition 2. I would ask the authors to either reformulate Eq. (21a) as a well-defined ratio-of-expectations estimator with its own properties, or to compute the true expectation of Eq. (14), and to validate the standard-error formulas on a genuinely stochastic MQU where the denominator varies."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nYou should know about this paper because it proposes a genuinely new object—a Sobol' matrix whose off-diagonal entries attribute correlations between outputs to specific inputs. That fills a real gap in the GSA literature: previous multi-output extensions stopped at the diagonal or scalar indices. The moment-based formulas for standard errors are also new, and the GP benchmark is extensive and mostly well-designed. The practical numbers look good.\n\nThe problem is in Section 5. The authors define the closed Sobol' matrix S_m as a random variable (Eq. 14), then claim its expectation is the ratio of expectations (Eq. 21a), with the intermediate claim E[sqrt(X)] = sqrt(E[X]) (Eq. 20b). Both are false unless the denominator D_M[y_M] is constant. For a genuine MQU—e.g., a GP with input-dependent variance—the denominator varies with the undetermined input, so Jensen's inequality gives strict inequality. The quantity they compute is not the expectation of the Sobol' matrix they defined. The benchmark cannot detect this because the ground truth is an MNU, where the denominator is deterministic and the two definitions coincide. This is a load-bearing flaw: the paper's main computational claim concerns a different estimator than the one defined.\n\nThe Section 2 justification that L^1 integrability implies integrability of all powers on a finite measure is backwards; the cited reference actually gives the opposite inclusion. This is easily fixed by assuming finite fourth moments, but as written the covariance-of-variances tensor W_mm' in Eq. (24) is not guaranteed to exist.\n\nMinor point: the benchmark includes a filtering step that removes GPs producing out-of-range indices; this can inflate apparent accuracy, and the authors should report results with and without filtering.\n\nMy take: the idea is worth pursuing and the practical results are suggestive, but the theoretical core needs substantial revision. The authors should either (a) redefine the computed statistic as a ratio-of-expectations summary and retract the claim that it is E[S_m], or (b) derive the actual expectation under stronger moment assumptions. As it stands, the paper is not ready for publication, but it is not a waste of time. I would send it to a serious referee who understands delta methods and Jensen's inequality, because the framework could become a useful tool after the fix.\n\nRecommendation: accept for peer review, with major revision expected.","headline":"Useful extension of Sobol' indices to multi-output correlations, but the central expected-value identity is false and needs correction before the paper can be trusted.","tokens_in":20366,"tokens_out":5254,"would_cite":false,"duration_ms":52368,"reading_group":"yes","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["60G07","60G15","62J10","62H99"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes a Sobol' matrix that measures how strongly each set of inputs drives the correlations between multiple outputs, with the classical Sobol' indices on the diagonal.","keywords":["global sensitivity analysis","Sobol' indices","multi-output models","output correlation","multi-output Gaussian process","uncertainty quantification","variance-based sensitivity","moment-based estimation"],"falsifier":"Take a model on $[0,1]^{M+1}$ with a heavy-tailed undetermined input, for example $y(u)=u_0+u_{\\bar M}^{-1/4}$, which has finite mean but infinite fourth moment, and compare the Taylor standard error of Equation (26) against a large Monte Carlo estimate of the true standard deviation of the Sobol' matrix entry; the formula should fail to track the simulation, showing that the moment-existence assumption is doing real work.","tokens_in":19362,"feed_emoji":"📊","tokens_out":9415,"duration_ms":90400,"temperature":0.7,"pith_summary":"The paper attempts to establish a multi-output generalization of Sobol' sensitivity analysis. Instead of one number per input-output pair, it defines, for every set of input axes $m$, a closed Sobol' matrix $S_m := V_m[y_m]/(D_M[y_M] \\otimes D_M[y_M])$ whose diagonal entries are exactly the classical Sobol' indices and whose off-diagonal entries measure how much that input set contributes to the correlation between two outputs. If the construction works, an analyst could ask not only which inputs matter for each response, but also which inputs change the linkage between responses, such as the trade-off between cost and efficacy or yield and purity. The paper further claims that both the matrix and its standard error are computable from the first two moments of a multi-output stochastic model, and reports benchmarks suggesting that a Gaussian-process surrogate needs only a few hundred samples to estimate matrix entries to about $0.02$.","feed_headline":"Sobol' matrices trace which inputs drive output correlations","feed_subtitle":"The diagonal keeps classical Sobol' indices; off-diagonal entries expose inputs behind output correlations.","key_machinery":"The central object is the closed Sobol' matrix $S_m := V_m[y_m]/(D_M[y_M] \\otimes D_M[y_M])$, where $y_m$ is the reduced model obtained by marginalizing inputs outside $m$, and the denominator is the outer product of output standard deviations of the full model. The argument is carried by representing the model as a multi-output model with quantified uncertainty, i.e. a stochastic process on $[0,1]^{M+1}$ with one undetermined uniform input appended to the determined inputs; marginalization and variance computation are then both expectations, so Fubini's theorem lets the paper evaluate them in the order that produces moments of the process. The first moment gives the Sobol' matrix itself, and the covariance of variances (the fourth-order tensor $W_{mm'}$, Equation (24)) feeds a Taylor-series error propagation formula that yields the standard error $T_m$. The benchmark compares these moment-based matrices against analytically computed matrices for test functions, using a Gaussian-process surrogate as the source of moments.","core_discovery":"The central claim is that the tensor extension of the scalar Sobol' index is the closed Sobol' matrix of Equation (14), defined through the coefficient-of-determination form rather than by elementwise division of covariance matrices, which can hit zero denominators. Each entry lies in $[-1,1]$; the diagonal reproduces the classical Sobol' index for each output, and an off-diagonal entry $[S_m]_{l,l'}$ expresses the fraction of covariance between outputs $l$ and $l'$ that input axes $m$ account for, with negative values when those inputs push the two outputs in opposite directions. Existence and uniqueness are argued by embedding models as multi-output models with quantified uncertainty, and the matrix plus its standard error are reduced to the first two moment tensors of that stochastic process using Fubini's theorem and the Kolmogorov extension theorem, with the standard error obtained by a Taylor expansion of the covariance of marginal variances.","pith_inferences":["Beyond the paper, the same two-moment reduction could be applied to any surrogate that supplies moment tensors, not only Gaussian processes; this would let the method scale to larger output counts by choosing a surrogate with cheap moment arithmetic.","The signed off-diagonal structure suggests a natural extension the authors do not pursue: for each input axis, the Sobol' matrix can be read as a signed network on outputs, allowing sensitivity analysis to double as a screening tool for which inputs reconfigure the correlation structure.","The noise-invariance result is proven for homoskedastic noise, and the benchmarks only use Gaussian noise; whether the cancellation survives heteroskedastic or non-Gaussian uncertainty is an open question that a direct test would settle.","Adding an explicit finite-fourth-moment assumption would make the standard-error formulas unconditional; without it, the reported error bars rest on the reversed inclusion argument flagged above."],"forward_implications":["The diagonal of every closed Sobol' matrix recovers the classical scalar Sobol' indices, so the matrix is a direct generalization rather than a separate sensitivity measure.","The sign and magnitude of off-diagonal entries let an analyst tell which inputs strengthen a correlation between outputs and which inputs oppose it, pointing to which inputs to vary in order to change a trade-off.","Because noise with constant variance cancels from the moment formula, Sobol' matrices and their standard errors are largely unaffected by additive output noise; the benchmarks still degrade when noise is so strong that the Gaussian-process surrogate itself stops predicting well.","Total Sobol' matrices $S^T_{M-m}=S_M-S_m$ give a dimension-reduction criterion: input axes whose total matrix is near zero can be dropped, and the benchmark's total-index accuracy is nearly independent of $m$ up to seven input dimensions.","The moment-based standard errors act as a diagnostic of numerical stability, with benchmark standardized scores mostly below 3 at the median and below 6 at the 90th percentile."],"supporting_citations":[{"why":"Supplies the original scalar Sobol' index that the closed Sobol' matrix generalizes.","marker":"[41]"},{"why":"Gives the classical high-dimensional model representation construction the paper extends to matrix form.","marker":"[42]"},{"why":"Introduces Sobol' indices as random variables for Gaussian process metamodels and supplies one of the benchmark test functions.","marker":"[31]"},{"why":"Supplies the Gaussian process formulation and kernel machinery used to compute the moment tensors in the benchmark.","marker":"[32]"},{"why":"Kolmogorov extension theorem, cited to establish that a multi-output model with quantified uncertainty is a stochastic process with well-defined finite-dimensional distributions.","marker":"[34]"},{"why":"Fubini's theorem, used to reverse the order of expectations so Sobol' matrices and errors reduce to moment tensors.","marker":"[49]"},{"why":"Cited for the claim that integrability on a finite measure implies all powers are integrable; this is the load-bearing justification for the fourth-moment assumption behind the standard errors.","marker":"[47]"},{"why":"One of the scalar test functions used to build the nine-output benchmark model.","marker":"[22]"},{"why":"Another scalar test function used to build the benchmark model with known analytical Sobol' matrices.","marker":"[37]"}],"fun_headline_variants":["Matrix Sobol' indices reveal input-output link directions","Sobol' matrices with quantified uncertainty for correlated outputs","Input relevance across outputs: Sobol' matrix with error bars","Multi-output sensitivity: Sobol' matrix and standard errors","From scalar Sobol' to matrices: track input drivers of covariance"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the model's random outputs are well-behaved enough that the average of their fourth powers is finite, a premise the paper justifies with the claim that integrability on a finite space implies all powers are integrable--which is backwards.","fun_headline_variants_meta":{"raw":{"variants":["Matrix Sobol' indices reveal input-output link directions","Sobol' matrices with quantified uncertainty for correlated outputs","Input relevance across outputs: Sobol' matrix with error bars","Multi-output sensitivity: Sobol' matrix and standard errors","From scalar Sobol' to matrices: track input drivers of covariance"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000362,"raw_usage":{"total_tokens":1902,"prompt_tokens":839,"completion_tokens":1063,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":455,"completion_tokens_details":{"reasoning_tokens":980}},"tokens_in":455,"tokens_out":1063,"duration_ms":8641,"temperature":1.0,"reasoning_tokens":980,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T21:28:02.730258+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a model on $[0,1]^{M+1}$ with a heavy-tailed undetermined input, for example $y(u)=u_0+u_{\\bar M}^{-1/4}$, which has finite mean but infinite fourth moment, and compare the Taylor standard error of Equation (26) against a large Monte Carlo estimate of the true standard deviation of the Sobol' matrix entry; the formula should fail to track the simulation, showing that the moment-existence assumption is doing real work.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the original scalar Sobol' index that the closed Sobol' matrix generalizes."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the classical high-dimensional model representation construction the paper extends to matrix form."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces Sobol' indices as random variables for Gaussian process metamodels and supplies one of the benchmark test functions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Gaussian process formulation and kernel machinery used to compute the moment tensors in the benchmark."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Kolmogorov extension theorem, cited to establish that a multi-output model with quantified uncertainty is a stochastic process with well-defined finite-dimensional distributions."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Fubini's theorem, used to reverse the order of expectations so Sobol' matrices and errors reduce to moment tensors."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Cited for the claim that integrability on a finite measure implies all powers are integrable; this is the load-bearing justification for the fourth-moment assumption behind the standard errors."},{"cited_title":"Ishigami and T","cited_arxiv_id":null,"evidence_quote":"One of the scalar test functions used to build the nine-output benchmark model."},{"cited_title":"design and estimator for the total sensitivity index , 181 (2010-02), no","cited_arxiv_id":null,"evidence_quote":"Another scalar test function used to build the benchmark model with known analytical Sobol' matrices."}],"review_version":1}