{"id":"ae78e8cf-7f78-4b5f-92eb-7385b12eb2fa","arxiv_id":"2411.13396","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"The paper derives Shapley value sensitivity formulas and efficient sampling algorithms for policy-augmented graphical hybrid models of biomanufacturing processes.","lead":"This paper presents a Shapley value sensitivity analysis framework for policy-augmented graphical hybrid models of biomanufacturing, with exact formulas for linear Gaussian models and an efficient permutation sampling method. The work aims to help process engineers identify which random factors, policy settings, and model parameters most influence production outcomes.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 1 and Theorem 2 silently assume that residual components are mutually independent; under the general Gaussian covariance the paper explicitly permits, the claimed exact Shapley value formulas are not correct.","rationale":"The reader's weakest assumption is the external validity of the linear Gaussian pKG approximation to the real bioprocess. That is a legitimate concern, but it is a modeling assumption that the paper explicitly inherits from Zheng et al. (2023) and presents as an approximation. My reading identified a more internal and more immediately load-bearing issue: the exactness of Theorems 1 and 2 depends on an unstated mutual-independence assumption on the residual random factors. The paper defines eee as multivariate Gaussian and treats its covariance matrix as a model parameter, yet the proof of Theorem 1 and the variance masking formula in Theorem 2 are only valid when residuals are independent. Since the exact SV formula for random factors is one of the paper's headline contributions and feeds directly into the linear-Gaussian computational-efficiency claims, this is the weakest point in the central argument. The good-faith reading also shows that the paper's general nonlinear pKG example from SDE discretization does use independent Brownian increments, so the intended setting may well be independent residuals; in that case the fix is to state the assumption explicitly and restrict the covariance structure accordingly, not to abandon the framework. For that reason the appropriate verdict remains CONDITIONAL: the paper should add the independence assumption (or correct the formulas for correlated residuals) and re-verify the relevant derivations and experiments under that assumption. I disagree with the reader only in locating the load-bearing risk: the more pressing correctness check is the covariance/independence gap inside the linear Gaussian analysis, not the external approximation error of the linear Gaussian model.","tokens_in":36393,"tokens_out":6227,"duration_ms":77864,"concrete_test":"Build a two-time-step scalar linear Gaussian pKG with n=m=1, β=0, reward r_2 = s_2, and residuals (e_1,e_2) jointly normal with mean zero, unit variances, and correlation ρ = 0.5. Compute the exact SV of e_1 at a fixed value x by enumerating all subsets U ⊆ {e_1,e_2} under the paper's value function g(U) = E[s_2|U], and compare it with the Theorem 1 expression. If the two disagree for ρ ≠ 0, the theorem relies on an unstated independence assumption; then re-run the same check with a diagonal covariance matrix to confirm that the formula is correct in the independent case.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim of Section 4 is that Theorem 1 gives the exact SV of a random factor e^k_h on the expected cumulative reward for linear Gaussian pKG models. This claim is load-bearing because it is advertised as an exact formula and is the basis for the claimed computational savings in the linear case. The proof in EC.1.3 computes the marginal contribution E[s_{t+1}|U ∪ {o}] - E[s_{t+1}|U] and asserts it equals R_{h,t} e^k_h 1_k, dropping any dependence on the conditional distribution of e^k_h given the other fixed residuals in U. That step is valid only if e^k_h is independent of every other residual component. The model statement in Section 2.1.2, however, only says that eee = (eee_1,...,eee_H) is multivariate Gaussian and explicitly includes its covariance matrix in the model parameters www; it does not impose independence. If residuals are correlated, then for a subset U not containing o, E[s_{t+1}|U] contains the conditional mean of e^k_h (and of other unfixed residuals) given U, so the marginal contribution is R_{h,t}(e^k_h - E[e^k_h|U]) plus additional propagated terms, not simply R_{h,t} e^k_h. Theorem 2 has the same issue: its variance formula uses V_{t+1} ⊙ (1/2(1^l + 1^{l⊤})), which is the conditional covariance only when conditioning on a subset of independent Gaussian variables; with correlation, one must use a Schur complement of the covariance matrix on the unfixed coordinates. A concrete scalar counterexample is H=2, n=1, β=0, reward r_2 = s_2, and (e_1,e_2) jointly normal with mean zero and covariance ρ ≠ 0. The exact SV of e_1 at fixed value x is (ρ/σ_1^2) x, while the Theorem 1 formula gives x (up to the reward coefficient). Thus either the paper must explicitly assume independent residual components across time and across state dimensions, or the exact-formula claims in the abstract and Section 4 require correction.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper develops a Shapley-value-based sensitivity analysis framework for policy-augmented knowledge graphical (pKG) hybrid models used in biomanufacturing. The framework defines value functions for three input types (random factors, policy parameters, and model parameters) and uses permutation sampling with pathway reuse to estimate Shapley values. For linear Gaussian pKG models, the authors derive closed-form expressions for the Shapley values of random factors (Theorems 1 and 2) and propose recursive algorithms that reduce the computational cost by O(H) for predictive analysis and O(H^3) for variance-based analysis. They also introduce TFWW-VRT, a permutation sampling algorithm combining the TFWW hypersphere transformation with randomized quasi-Monte Carlo and antithetic sampling, and they empirically compare it with BMT and SCT. The paper includes proofs of estimator properties, complexity analyses, and a publicly available code repository.","tokens_in":36939,"tokens_out":8403,"duration_ms":90655,"significance":"If correct, the linear-Gaussian results provide a substantial computational improvement for sensitivity analysis in heavily instrumented bioprocesses: the O(H) and O(H^3) reductions are practically meaningful, and the TFWW-VRT method appears to improve estimation accuracy at no extra cost. The paper also contributes a clean formulation of Shapley-value sensitivity analysis for a class of hybrid stochastic dynamic models, with attention to model uncertainty through the posterior distribution of parameters. The authors are to be credited for supplying full proofs of the estimator properties (Propositions 1 and 2), complexity statements (Propositions 3-5), and for making the source code and data publicly available. However, the central exact-formula claim (Theorems 1 and 2) is currently only valid under an unstated independence condition, and the empirical validation does not directly test these formulas. The practical relevance of the linear Gaussian approximation for the real fermentation process is also not assessed.","major_comments":[{"comment":"Theorem 1 is not correct for the general multivariate Gaussian residuals permitted by the model statement in Section 2.1.2. The proof in EC.1.3 computes the marginal contribution as E[s_{t+1}|U∪{o}] - E[s_{t+1}|U] = R_{h,t} e^k_h 1_k, but this step holds only if e^k_h is independent of every other residual component. Since the covariance matrix of eee is part of the model parameters and is not assumed diagonal, conditioning on a subset U changes the conditional means of the unfixed residuals when they are correlated with those in U. A concrete counterexample is H=2, n=1, β^s_1=β^a_1=0, reward r_2=s_2, and (e_1,e_2) jointly Gaussian with mean zero and covariance ρσ_1σ_2; the Shapley value of e_1 on r_2 is (1/2)(σ_2/σ_1)ρ e_1, while Theorem 1 gives 0. The same independence issue affects Theorem 2: the expression V_{t+1} ⊙ (1/2(1^l+1^{l⊤})) equals the conditional covariance only when the excluded coordinates are independent of the included ones; for correlated residuals, a Schur complement of the covariance matrix on the unfixed coordinates is required. The paper either needs to state an explicit independence assumption for the residuals (which would make the cross-covariance terms in Theorem 2 vanish) or derive the correct conditional-expectation/conditional-variance formulas for the general Gaussian case.","section":"Section 4.1 / EC.1.3"},{"comment":"The empirical study does not validate Theorems 1 and 2. Figure 4 reports MSE for policy parameters only, and Figures 5-9 show Shapley value point estimates for random factors without any error bars or comparison to a ground truth. Given that these theorems are the central analytical contribution, the paper should verify them on a small linear Gaussian pKG model by comparing the formula against exhaustive enumeration of the Shapley value definition in Equation (7), ideally for both diagonal and non-diagonal residual covariance matrices. The sample sizes Q, D, and L used in the experiments are also not reported for the results in Figures 5-9, which makes the empirical results difficult to reproduce or interpret.","section":"Section 6.4"},{"comment":"The linear Gaussian pKG model is introduced as an approximation of the nonlinear bioprocess on a fast monitoring time scale, but the approximation error is never quantified. The linear Gaussian case study uses synthetic data generated according to the linear model itself, so it cannot establish that the linear structure faithfully represents the real fermentation dynamics. Without an error analysis or a comparison to the nonlinear model on real experimental data, the practical significance of the exact Shapley value formulas for the biomanufacturing process remains unclear. The paper should at least discuss the sensitivity of the SV estimates to the linearization error and ideally provide an empirical assessment of this error.","section":"Section 2.1.2 / Section 6.4"}],"minor_comments":[{"comment":"The caption of Figure 8 states 'SV of selected policy parameters and outputs in linear pKG model', but the text and the heatmap describe random factors; the caption should say 'random factors'.","section":"Figure 8"},{"comment":"The header 'Discrepency' in Table 1 is a typo and should be 'Discrepancy'.","section":"Table 1"},{"comment":"In Section 6.3 the random factors for the predictive analysis are fixed at e^k_h = E[e^k_h] + σ(e^k_h); for nonlinear models the marginal contribution depends on the chosen conditioning value, and the paper should state whether the reported results are sensitive to this choice.","section":"Section 6.3"},{"comment":"The notation in Algorithm 3, especially the use of c^{(k)} in Step 1 of the TFWW transformation in Appendix EC.2.3, is dense; a short worked example for a small s would improve readability.","section":"Algorithm 3"}],"recommendation":"major_revision","confidential_remarks":"The main issue is the unstated independence assumption in Theorems 1 and 2. I believe the intended application (SDE discretization) suggests independent noise increments, and adding that assumption would make the formulas correct; however, as written the claim of exactness for general multivariate Gaussian residuals is false. The paper would also be strengthened by a direct numerical verification of the exact formulas and by a more transparent reporting of experimental sample sizes. The paper fits well within the scope of a journal targeting statistical machine learning or operations research applications."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper has a genuinely useful algorithmic core, but the headline exact formula for Shapley values of correlated Gaussian residuals is not correct as stated. That is the thing to know before sending anyone to the appendix.\n\nWhat is new: exact SV formulas for random factors in linear Gaussian pKG models (Theorems 1 and 2), an O(H) / O(H^3) speedup via recursive calculation of pathway coefficients, and a permutation sampler (TFWW-VRT) combining the TFWW sphere mapping with randomized QMC and antithetic sampling. The sampler is tested cleanly against BMT/SCT: lower discrepancy, same O(Ds) cost, and an MSE plot that shows a real gain from the variance reduction. The GitHub link is live, which is more than many papers in this space. The nonlinear-pKG empirical study uses real fermentation data and the interpretation of the SV heatmaps is sensible.\n\nThe soft spot is load-bearing. Section 2.1.2 defines the residual vector as multivariate Gaussian with an arbitrary covariance matrix. The proof of Theorem 1 (EC.1.3) computes E[s_{t+1}|U∪{o}] - E[s_{t+1}|U] as R_{h,t} e^k_h 1_k for every subset U. That is valid only if e^k_h is independent of every other residual component. Under correlated residuals, the conditional mean of e^k_h given U is a linear combination of the fixed components of U, and the difference between the two conditional expectations is R_{h,t}(e^k_h - E[e^k_h|U]) plus propagated terms. A two-step scalar example with correlation ρ gives SV = (ρ/σ_1^2)x, not x. Theorem 2 has the same problem: the mask-matrix formula for the conditional covariance is the independent-coordinates formula; with correlation you need a Schur complement. So the exactness claim in the abstract and Section 4 is overstated unless the model is restricted to diagonal covariance or the formulas are re-derived.\n\nSecondary issues: the SV heatmaps carry no error bars, the linear case study is synthetic, and sample sizes (Q, D, L) are not always reported. Those are fixable. The linear Gaussian approximation to the bioprocess is assumed, following Zheng et al. (2023), without validation here; that is a standard practice but worth stating.\n\nWho this is for: operations researchers working on sensitivity analysis for stochastic dynamic systems, especially biomanufacturing. The algorithmic core (pathway reuse, TFWW-VRT) is solid and testable. The theorems need correction before the paper is reliable.\n\nRecommendation: send it to peer review, but explicitly ask the referee to check the independence assumption in the proofs. If the authors add a diagonal-covariance assumption or fix the theorems with Schur complements, this could be a useful paper. As it stands, the central exact-formula claim does not hold for the general Gaussian model they advertise.","headline":"Useful sampler and computational tricks, but the exact Shapley formula for correlated Gaussian residuals is wrong as stated.","tokens_in":37366,"tokens_out":3885,"would_cite":false,"duration_ms":40860,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"For linear Gaussian policy-augmented knowledge-graph models of bioprocesses, the Shapley value of each random factor on expected cumulative reward has an exact closed form, and a recursive computation plus variance-reduced permutation…","keywords":["Biomanufacturing","sensitivity analysis","Shapley value","Bayesian dynamic networks","quasi-Monte Carlo","simulation","linear Gaussian approximation","variance reduction"],"falsifier":"Measure the wall-clock time of Algorithm EC.2 for variance-based SV on the linear Gaussian pKG as $H$ goes from 4 to 32 at fixed $Q$ and $D$: the claimed $O(H^3)$ speedup predicts the runtime should grow roughly 8-fold when $H$ doubles; if it grows 64-fold (the brute-force rate), the reuse argument fails.","tokens_in":36202,"feed_emoji":"🧬","tokens_out":7901,"duration_ms":72797,"temperature":0.7,"pith_summary":"The paper argues that Shapley value (SV) sensitivity analysis, which fairly attributes output differences and risks to input variables, can be made practical for policy-augmented knowledge-graph (pKG) hybrid models of biomanufacturing processes. For the common case where a heavily instrumented process is monitored faster than its dynamics, a linear Gaussian pKG model applies, and the paper proves an exact SV formula for random shocks (Theorem 1) and gives recursive algorithms that cut the computational cost of estimating SVs of policy and model parameters by factors of $O(H)$ and $O(H^3)$. It also introduces TFWW-VRT, a permutation-sampling scheme that combines the TFWW hypersphere transformation with randomized quasi-Monte Carlo and antithetic sampling, yielding lower estimation error than existing BMT/SCT sampling at the same budget. If these claims hold, researchers and practitioners can rank the variation sources, control settings, and uncertain parameters that most affect product quality and profit, and respond faster in process development and online control.","feed_headline":"Linearized bioprocess models get exact, fast Shapley sensitivity","feed_subtitle":"Rank which random shocks and control knobs move bioprocess output, with provable speedups and better sampling.","key_machinery":"The load-bearing objects are: the pKG model, a Bayesian-network-style hybrid model whose nodes are process states, actions from a parametric policy, rewards, and random shocks; the linear Gaussian approximation of the state transition, reward, and policy; the pathway coefficient $R_{h,t} = \\prod_{j=t}^{h} ((\\beta^s_j)^\\top + (\\beta^a_j)^\\top \\theta^\\top_j)$ that propagates a shock at period $h$ to period $t+1$; the value functions $g(U)$ defined either as conditional expectation or expected remaining variance of the output given a subset of inputs; and the permutation estimator of Shapley values (Castro's ApproShapley). The machinery that carries the efficiency argument is the recursive reuse of these pathway products and of block covariance terms, which turns sums over long horizons into a short recurrence; the sampling machinery is the TFWW-VRT permutation sampler, which maps low-discrepancy points from a unit cube to a hypersphere, orders coordinates to get permutations, and appends reversed permutations as antithetic pairs.","core_discovery":"The central claim is that for linear Gaussian pKG models, the SV of a single random factor $e^k_h$ on the expected cumulative reward is exactly $E_w[\\sum_{t=0}^{H-1} (b^\\top_{t+1}\\theta^\\top_{t+1}+c^\\top_{t+1}) R_{h,t} e^k_h 1_k]$, i.e., the propagated pathway coefficient product times the shock; the paper proves this as Theorem 1 and its variance-analog as Theorem 2. For the SVs of policy parameters and model parameters, where no closed form exists, the paper proves that recursive reuse of intermediate pathway and covariance computations reduces the cost of the SV estimates from $O(QDH^3(n^4m+n^3m^2))$ to $O(QDH^2(n^4m+n^3m^2))$ for predictive analysis and from $O(QDH^6(n^4m+n^3m^2))$ to $O(QDH^3(n^4m+n^3m^2))$ for variance-based analysis. It further claims its TFWW-VRT sampler generates more uniform permutations than BMT and SCT with linear-in-dimension cost, and that with randomized QMC and antithetic pairs it achieves smaller MSE in SV estimation than the baselines.","pith_inferences":["A direct diagnostic falls out of the exact formula: because the SV of $e^k_h$ is linear in the product of pathway matrices and the shock value, the dominant sensitivity path can be read off term-by-term, which could be used to identify which intermediate transition step amplifies a given variation most.","TFWW-VRT is not tied to bioprocesses; it could be dropped into any Shapley-value estimation pipeline (e.g., feature attribution in ML models) where permutation sampling is the bottleneck, provided the same uniformity gains transfer.","The paper's complexity savings assume the linear-Gaussian approximation; a natural extension would be to adapt the recursive reuse to piecewise-linear or locally linearized pKG models, preserving some speedup beyond the exact linear regime.","A falsifiable prediction of the approach is that, on a real fed-batch data set with enough trajectories, the linear-Gaussian Shapley rankings should match the simulation-based rankings from the nonlinear model on coarse qualitative patterns (e.g., which shocks matter); if they diverge sharply, the fast-monitoring approximation is not adequate."],"forward_implications":["Sensitivity rankings of random shocks, policy gains, and model parameters for a linearized bioprocess can be computed exactly or with rigorous error bounds rather than by brute-force simulation, so sensitivity checks can run in time for real-time process control.","The $O(H^3)$ speedup for variance-based SV means that processes with hundreds of monitored time steps, previously intractable, become routine; the same structure applies to any linear-Gaussian policy-augmented dynamic network, not only fermentation.","The TFWW-VRT sampler with its variance reduction gives smaller SV estimation error at fixed permutation budget, so the number of model simulations needed to reach a target accuracy can be cut substantially.","Model-parameter SVs expose which uncertain kinetic constants most inflate output variance, guiding where to spend scarce lab experiments."],"supporting_citations":[{"why":"Defines the pKG hybrid model and its linear Gaussian approximation that the paper builds on; the model and data are inherited from this work.","marker":"Zheng et al. (2023)"},{"why":"Introduces Bayesian network hybrid modeling with SV-based sensitivity analysis on linear Gaussian models, the direct precedent this paper generalizes.","marker":"Xie et al. (2022)"},{"why":"Supplies the permutation-sampling Shapley estimator (ApproShapley) that underlies the paper's SV estimators and Proposition 1.","marker":"Castro et al. (2009)"},{"why":"Provides the BMT/SCT hypersphere permutation sampling baselines that TFWW-VRT improves on and the discrepancy metric used for comparison.","marker":"Mitchell et al. (2022)"},{"why":"Supplies the TFWW hypersphere transformation used to generate uniform points from the unit cube for permutation sampling.","marker":"Fang and Wang (1993)"},{"why":"Source of the randomized quasi-Monte Carlo (randomized Sobol' sequence) used in Step 1 of TFWW-VRT.","marker":"L'Ecuyer (2018)"},{"why":"Source of the antithetic sampling variance-reduction technique used in Step 4 of TFWW-VRT.","marker":"Lemieux (2009)"},{"why":"Establishes Shapley value sensitivity analysis for variance components, the conceptual foundation for using SV as a sensitivity index.","marker":"Owen (2014)"}],"fun_headline_variants":["Exact Shapley values for linear bioprocess models","Recursive Shapley speeds up bioprocess sensitivity","Better sampling for Shapley sensitivity in bioprocesses","Shapley-based sensitivity analysis for bioprocess models","SV-pKG: exact, fast sensitivity for bioprocess control"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole edifice of exact formulas and complexity savings rests on the assumption that a linear Gaussian pKG model (linear transitions, linear reward, linear policy, Gaussian noises) faithfully approximates the real bioprocess on the fast monitoring time scale; if that approximation is poor, the exact formulas and gains do not transfer to the true process.","fun_headline_variants_meta":{"raw":{"variants":["Exact Shapley values for linear bioprocess models","Recursive Shapley speeds up bioprocess sensitivity","Better sampling for Shapley sensitivity in bioprocesses","Shapley-based sensitivity analysis for bioprocess models","SV-pKG: exact, fast sensitivity for bioprocess control"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000641,"raw_usage":{"total_tokens":2985,"prompt_tokens":1018,"completion_tokens":1967,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":634,"completion_tokens_details":{"reasoning_tokens":1892}},"tokens_in":634,"tokens_out":1967,"duration_ms":14851,"temperature":1.0,"reasoning_tokens":1892,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:26:37.358113+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the wall-clock time of Algorithm EC.2 for variance-based SV on the linear Gaussian pKG as $H$ goes from 4 to 32 at fixed $Q$ and $D$: the claimed $O(H^3)$ speedup predicts the runtime should grow roughly 8-fold when $H$ doubles; if it grows 64-fold (the brute-force rate), the reuse argument fails.","supporting_citations":[{"cited_title":"INFORMS Journal on Computing 35(1):66--82","cited_arxiv_id":null,"evidence_quote":"Defines the pKG hybrid model and its linear Gaussian approximation that the paper builds on; the model and data are inherited from this work."},{"cited_title":"Naval Research Logistics 69(3):461--483","cited_arxiv_id":null,"evidence_quote":"Introduces Bayesian network hybrid modeling with SV-based sensitivity analysis on linear Gaussian models, the direct precedent this paper generalizes."},{"cited_title":"Computers & Operations Research 36(5):1726--1730","cited_arxiv_id":null,"evidence_quote":"Supplies the permutation-sampling Shapley estimator (ApproShapley) that underlies the paper's SV estimators and Proposition 1."},{"cited_title":"Journal of Machine Learning Research 23(43):1--46","cited_arxiv_id":null,"evidence_quote":"Provides the BMT/SCT hypersphere permutation sampling baselines that TFWW-VRT improves on and the discrepancy metric used for comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the TFWW hypersphere transformation used to generate uniform points from the unit cube for permutation sampling."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Source of the randomized quasi-Monte Carlo (randomized Sobol' sequence) used in Step 1 of TFWW-VRT."},{"cited_title":"Springer Series in Statistics, 1 edition","cited_arxiv_id":null,"evidence_quote":"Source of the antithetic sampling variance-reduction technique used in Step 4 of TFWW-VRT."},{"cited_title":"SIAM/ASA Journal on Uncertainty Quantification 2(1):245--251","cited_arxiv_id":null,"evidence_quote":"Establishes Shapley value sensitivity analysis for variance components, the conceptual foundation for using SV as a sensitivity index."}],"review_version":1}