{"id":"1793219e-66f3-412d-bc31-e5de9999a604","arxiv_id":"2607.05809","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Nested-simulation view of Sobol' index estimation yields MSE rates, optimal outer/inner budgets, and jackknife/LHS guidance for pick-freeze-style estimators.","lead":"This paper unifies pick-freeze and nested-simulation estimators of Sobol' indices under a shared budget, and shows when jackknife bias correction and Latin hypercube sampling help or hurt. Practitioners get clear MSE rates and budget-allocation rules for choosing among standard nested, unbiased jackknife, and split-jackknife estimators under CMC and LHS.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.5","headline":"The claimed CMC rate separation for the split jackknife rests on nested-simulation moment and growth conditions that are assumed, not derived from primitive properties of a general black-box simulator; if they fail the rate advantage need not hold.","rationale":"The reader's weakest_assumption correctly isolates the soft underbelly of the central rate claim. Within the stated nested-simulation framework the comparative argument (pick-freeze as fixed-m nested, bias-variance trade-offs, allocations, and the LHS interaction) is internally consistent and of the expected asymptotic-Monte-Carlo type; no hidden circularity, double-counted budget for the preliminary mean sample, or broken mapping of pick-freeze dependence is forced by the abstract and structured claims. The moment/growth conditions are standard in the nested-simulation literature and are acknowledged as technical, so they do not constitute an internal inconsistency, yet they remain the least secure link for any claim of practical estimator superiority on general black-box simulators. Consequently the CONDITIONAL verdict (accept-shaped contribution pending verification of those conditions and fuller numerical/code availability) needs no adjustment. Confidence stays moderate because the stress test, like the reader's pass, is not a line-by-line proof check.","tokens_in":2179,"tokens_out":633,"duration_ms":77061,"concrete_test":"Construct a Sobol' test function whose conditional moments of the order required by the paper's assumptions are infinite (e.g., a g-function or Ishigami variant with power-law tails). Recompute empirical MSE versus total budget N for the standard nested, unbiased jackknife, and split jackknife estimators under the paper's recommended CMC allocations. If the split jackknife loses its clear O(N^{-1}) slope while the other two remain slower, the rate separation is fragile to the moment conditions.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The strongest claim is the CMC rate separation: under an appropriate outer/inner allocation the split jackknife nested estimator of the Sobol' numerator attains the canonical MSE rate O(N^{-1}), while the standard nested estimator and the unbiased jackknife attain only the slower nested-simulation rate; pick-freeze estimators are recovered as the fixed-m special case. That separation is obtained from bias and variance expansions that require finite higher moments of the conditional expectation μ(X)=E[Y|X] and of the inner-level estimator, together with m growing suitably with total budget N. These are imposed as technical regularity conditions in the CMC-rate and budget-allocation analysis rather than verified from assumptions on the simulator f. When the moment or growth hypotheses fail (heavy-tailed conditional moments, insufficient growth of m), the expansions that produce the O(N^{-1}) versus slower-rate distinction cease to be valid, so the claimed superiority of the split jackknife is conditional on unchecked hypotheses. The pick-freeze reinterpretation and the LHS caveats are secondary and less fragile; the rate separation is the load-bearing piece.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"This paper revisits Sobol' index estimation from a nested-simulation perspective. It shows that several classical pick-freeze estimators can be interpreted as nested simulation estimators with fixed inner-level sample sizes, enabling a unified bias–variance comparison under a common computational budget. Building on that view, the authors analyze the standard nested estimator of the Sobol' numerator and introduce two jackknife-based extensions: an unbiased jackknife estimator and a split jackknife estimator that uses an independent preliminary sample to estimate the mean. Under crude Monte Carlo they derive MSE rates and outer/inner budget allocations, claiming that the split jackknife attains the canonical rate O(N^{-1}) under an appropriate allocation while the standard nested and unbiased jackknife estimators attain only the slower nested-simulation rate. They further characterize the effect of Latin hypercube sampling, which can improve the standard nested estimator but may undermine jackknife bias reduction unless the inner sample size grows with total budget. Numerical experiments are used to corroborate the theory and to give practical guidance on estimator selection under CMC and LHS.","tokens_in":2356,"tokens_out":1253,"duration_ms":40302,"significance":"If the rate and allocation results hold under the stated conditions, the paper offers a useful methodological unification of the pick-freeze and nested-simulation literatures for Sobol' index estimation, together with a concrete recommendation favoring the split jackknife when a canonical MSE rate is desired under CMC. The explicit budget-allocation characterizations and the LHS caveats (including the warning that LHS can undo jackknife bias reduction) are practically relevant and go beyond a pure asymptotic exercise. The work is self-contained and methodological rather than application-driven; its main value is the rate separation, the nested reinterpretation of pick-freeze estimators, and the comparative guidance under a common budget. Numerical corroboration of the asymptotic claims is a clear strength.","major_comments":[{"comment":"The central CMC rate separation—that the split jackknife nested estimator of the Sobol' numerator attains MSE O(N^{-1}) under an appropriate outer/inner allocation, while the standard nested and unbiased jackknife estimators attain only the slower nested-simulation rate—rests on nested-simulation regularity (finite higher moments of the conditional expectation μ(X)=E[Y|X] and of the inner-level estimator, and suitable growth of the inner sample size m with total budget N). These conditions appear as technical hypotheses in the CMC-rate and budget-allocation analysis rather than as consequences of primitive assumptions on a general black-box simulator. The main rate theorem(s) should state these hypotheses explicitly and the surrounding discussion should briefly indicate when they may fail (e.g., heavy-tailed conditional moments, insufficient growth of m), so that the scope of the claimed","section":"CMC rate analysis and budget-allocation sections (split jackknife vs. standard nested / unbiased jackknife)"},{"comment":"The split jackknife uses an independent preliminary sample to estimate the mean. The total-budget accounting and the optimal outer/inner allocation that deliver the O(N^{-1}) rate must include the cost of that preliminary sample; if the preliminary sample is treated as free or is omitted from N, the claimed rate and allocation are not comparable to the other estimators under a common computational budget. Please make the budget identity and the resulting allocation explicit in the statement of the rate result and in the numerical design.","section":"Definition of the split jackknife estimator and associated CMC allocation analysis"}],"minor_comments":[{"comment":"In the pick-freeze-as-nested reinterpretation, state clearly for each classical estimator which fixed inner-level sample size m it corresponds to and whether the outer design is shared or independent, so that the common-budget comparison is unambiguous.","section":"Section establishing pick-freeze estimators as fixed-m nested estimators"},{"comment":"When discussing LHS, separate more sharply the effect on variance of the standard nested estimator from the effect on the bias expansion of the jackknife estimators; the abstract’s caveat that jackknife bias reduction can be undermined unless m grows with N should be mirrored by a short, explicit statement next to the corresponding theorem or proposition.","section":"LHS analysis section"},{"comment":"Notation for the Sobol' numerator, the conditional expectation μ(X), the inner estimator, and the total budget N should be fixed early and used consistently; occasional switches between “inner replications” and “inner sample size m” make the allocation formulas harder to parse.","section":"Notation and setup"},{"comment":"In the numerical experiments, report the precise outer/inner allocations used for each estimator (including the preliminary-sample size for the split jackknife) and the number of independent macro-replications, so that the MSE curves can be checked against the predicted rates.","section":"Numerical experiments"},{"comment":"A short related-work paragraph situating the jackknife constructions relative to existing bias-correction devices for nested estimation of Var(E[Y|X]) would help readers place the contribution.","section":"Introduction / related work"}],"recommendation":"minor_revision","confidential_remarks":"The manuscript is a solid methodological contribution in nested simulation / global sensitivity analysis and is a reasonable fit for a statistics methodology venue. The skeptic concern about unverified moment conditions is real but standard for this literature; I have framed it as a scope-clarification request rather than a correctness failure. I do not see load-bearing internal inconsistency. Novelty relative to the existing nested-simulation and pick-freeze literatures should be checked by the editor against recent concurrent work on jackknife nested estimators, but on the face of the manuscript the unified comparison and the split-jackknife rate claim look publishable after the clarifications above."},"author_rebuttal":{"model":"grok-4.5","summary":"We thank the referee for a careful and constructive report. The two major comments concern (i) explicit statement of the nested-simulation regularity conditions underlying the CMC rate separation and a brief discussion of their scope, and (ii) transparent total-budget accounting for the independent preliminary sample used by the split jackknife. Both points improve clarity and comparability under a common computational budget. We will revise the main CMC-rate and allocation statements, the surrounding discussion, and the numerical design accordingly. We believe these changes fully address the referee’s concerns while preserving the paper’s main contributions: the nested reinterpretation of pick-freeze estimators, the rate separation under CMC, the associated outer/inner allocations, and the LHS caveats.","responses":[{"response":"We agree. The CMC rate separation and the associated outer/inner allocations are established under nested-simulation regularity: finite higher moments of μ(X)=E[Y|X] and of the inner-level estimator, together with a suitable growth condition on the inner sample size m relative to the total budget N. In the revised manuscript we will state these hypotheses explicitly in the main CMC-rate theorem(s) (and in the corresponding allocation corollaries), rather than leaving them only in technical lemmas or intermediate arguments. We will also add a short discussion of scope: the O(N^{-1}) claim for the split jackknife can fail if conditional moments of μ(X) or of the inner estimator are infinite (heavy tails), or if m does not grow sufficiently with N so that the residual bias/variance terms do not become negligible at the claimed rate. We will note that these conditions are standard in the nested-simulation literature and are typically satisfied for simulators with bounded or light-tailed responses, while remaining assumptions on the black-box model rather than consequences of fully primitive structural hypotheses. No change is made to the rate statements themselves; the revision is one of explicitness and scope.","revision_made":"yes","referee_comment":"The central CMC rate separation—that the split jackknife nested estimator of the Sobol' numerator attains MSE O(N^{-1}) under an appropriate outer/inner allocation, while the standard nested and unbiased jackknife estimators attain only the slower nested-simulation rate—rests on nested-simulation regularity (finite higher moments of the conditional expectation μ(X)=E[Y|X] and of the inner-level estimator, and suitable growth of the inner sample size m with total budget N). These conditions appear as technical hypotheses in the CMC-rate and budget-allocation analysis rather than as consequences of primitive assumptions on a general black-box simulator. The main rate theorem(s) should state these hypotheses explicitly and the surrounding discussion should briefly indicate when they may fail (e.g., heavy-tailed conditional moments, insufficient growth of m), so that the scope of the claimed"},{"response":"We agree that comparability under a common computational budget requires that the cost of the independent preliminary sample be included in N. In the revised manuscript we will make the budget identity explicit in the statement of the split-jackknife CMC rate result and in the associated allocation analysis: if n_pre denotes the preliminary sample size used to estimate the mean, n the outer sample size, and m the inner sample size, then the total budget satisfies N = n_pre + n·m (up to the usual constant factors for the paired inputs in the Sobol' numerator), and the O(N^{-1}) rate is obtained under an allocation in which n_pre, n, and m all grow with N in a manner we will state explicitly (with n_pre of lower order than N when that is optimal, so that the preliminary cost does not dominate). We will likewise revise the numerical design so that reported budgets and MSE curves for the split jackknife count the preliminary sample, ensuring a fair comparison with the standard nested and unbiased jackknife estimators under the same N. If any current wording could be read as treating the preliminary sample as free, it will be corrected.","revision_made":"yes","referee_comment":"The split jackknife uses an independent preliminary sample to estimate the mean. The total-budget accounting and the optimal outer/inner allocation that deliver the O(N^{-1}) rate must include the cost of that preliminary sample; if the preliminary sample is treated as free or is omitted from N, the claimed rate and allocation are not comparable to the other estimators under a common computational budget. Please make the budget identity and the resulting allocation explicit in the statement of the rate result and in the numerical design."}],"tokens_in":1934,"tokens_out":980,"duration_ms":24974,"standing_objections":[]},"desk_editor":{"model":"grok-4.5","letter":"The one thing to know: this paper puts pick-freeze and nested Sobol' numerator estimators under a single computational budget and shows that a split jackknife nested estimator can hit the canonical O(N^{-1}) MSE rate under CMC, while the plain nested estimator and an unbiased jackknife stay at the slower nested rate. That comparative rate result, plus the fixed-m reading of classical pick-freeze, is the actual contribution.\n\nWhat is new is the framing, not a brand-new estimator family. Interpreting pick-freeze as nested simulation with fixed inner size lets them compare bias-variance and allocate outer/inner effort cleanly. The split jackknife (independent pilot mean) is the piece that recovers the canonical rate; they also spell out when LHS helps the standard nested estimator and when it undercuts jackknife bias reduction unless m grows with N. The abstract and structure match a coherent theory program, and the numerical section is there to check the rates and give practical selection guidance. Circularity is low: this is estimators, expansions, and rates, not fitting free constants.\n\nSoft spot, in proportion: the rate separation rests on nested-simulation regularity—finite moments of μ(X) and the inner estimator, and suitable growth of m with N. Those are stated as technical conditions, not derived from primitive properties of a general black-box f. If moments are heavy or m does not grow as assumed, the expansions that deliver O(N^{-1}) versus the slower nested rate stop applying. That is a real caveat, but it is the usual caveat in nested Monte Carlo theory, not a hidden crack in the argument. The pick-freeze reinterpretation and the LHS interaction notes are less fragile. I would not treat the stress-test as a reason to dismiss the central claim; I would treat it as “check the moment hypotheses before you trust the rate ranking on a new simulator.”\n\nThis is for people who already care about Sobol' indices, nested simulation, or budgeted Monte Carlo GSA. A general statistician will not reorganize their toolkit around it; a methods person in sensitivity analysis will get a usable comparison and allocation map. The math and citation pattern look like honest engagement with the nested-sim and pick-freeze literature. It deserves a serious referee, not a desk reject. Send it out; ask the referee to pressure-test the moment/growth conditions and the numerics, not to reinvent the framing.","headline":"Clean nested-simulation unification of pick-freeze and jackknife Sobol' estimators; the CMC rate separation is real under standard conditions, not a free lunch for black-box simulators.","tokens_in":3012,"tokens_out":595,"would_cite":false,"duration_ms":21867,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62G05","65C05","62F12"],"pacs":[],"model":"grok-4.5","headline":"A split jackknife nested estimator recovers the canonical O(N^{-1}) MSE rate for Sobol' index numerators under crude Monte Carlo.","keywords":["Sobol' indices","nested simulation","jackknife bias correction","budget allocation","Latin hypercube sampling","global sensitivity analysis","pick-freeze estimators","variance of conditional expectation"],"falsifier":"Under crude Monte Carlo with total budget N, plot log-MSE versus log-N for the split jackknife (optimal allocation) against the standard nested and unbiased jackknife estimators; if the split jackknife slope is not asymptotically -1 while the others remain steeper, the central rate claim is false.","tokens_in":3007,"feed_emoji":"📊","tokens_out":920,"duration_ms":13985,"temperature":0.7,"pith_summary":"This paper recasts classical Sobol' index estimation as a nested-simulation problem and shows that several standard pick-freeze estimators are simply nested estimators whose inner sample sizes are fixed by design. From that common vantage point the authors compare bias and variance under a shared computational budget N. They analyze the usual nested Monte Carlo estimator of the Sobol' numerator and introduce two jackknife corrections: an unbiased jackknife and a split jackknife that uses an independent pilot sample for the outer mean. Under crude Monte Carlo the split jackknife, with a suitable outer/inner allocation, attains the ordinary Monte Carlo MSE rate O(N^{-1}); the plain nested estimator and the unbiased jackknife remain stuck at the slower nested-simulation rate. The paper also maps how Latin hypercube sampling changes the picture: it can help the plain nested estimator yet can destroy the bias cancellation of the jackknife methods unless the inner size grows with N. The practical payoff is a clear ranking of estimators and sampling designs for global sensitivity analysis when every simulation call is expensive.","feed_headline":"Split jackknife restores the ordinary Monte Carlo rate for Sobol' indices","feed_subtitle":"Under crude Monte Carlo a pilot-sample jackknife beats standard nested and unbiased jackknife estimators.","key_machinery":"The split jackknife nested estimator: an outer sample of conditional expectations is estimated by independent inner replications; a leave-one-out jackknife corrects the squared-mean bias, while an independent pilot sample estimates the overall mean so that the jackknife correction does not re-introduce dependence that would spoil the rate. Optimal outer/inner allocation then converts the usual nested bias-variance trade-off into the ordinary Monte Carlo rate.","core_discovery":"Under crude Monte Carlo, the split jackknife nested estimator of the Sobol' index numerator attains the canonical MSE rate O(N^{-1}) under an appropriate outer/inner budget allocation, whereas the standard nested simulation estimator and the unbiased jackknife estimator attain only the slower nested-simulation rate. Several classical pick-freeze estimators are nested estimators with fixed inner-level sample sizes, so the same budget-allocation analysis applies to them directly.","pith_inferences":["The same split-jackknife construction may lift other nested estimators of variance-of-conditional-expectation functionals (e.g., conditional value-at-risk or nested risk measures) from the nested rate to the canonical rate.","When the black-box simulator itself admits a low-discrepancy or multilevel structure, combining the split jackknife with multilevel Monte Carlo could further reduce constants while preserving the O(N^{-1}) rate.","If moment conditions fail, diagnostic residual plots of the estimated conditional expectations against inner sample size could flag when the rate separation collapses and force a more conservative allocation."],"forward_implications":["Under a fixed simulation budget the split jackknife is the preferred nested estimator for Sobol' numerators when sampling is crude Monte Carlo.","Classical pick-freeze schemes can be ranked against nested estimators by treating their fixed inner sizes as special cases of the same budget allocation.","Latin hypercube sampling can improve the plain nested estimator but requires the inner sample size to grow with N if jackknife bias reduction is to be retained.","Practitioners gain concrete outer/inner allocation rules that convert the usual nested-simulation MSE into the ordinary Monte Carlo rate."],"fun_headline_variants":["Split jackknife attains O(N^{-1}) MSE rate for Sobol' under CMC","Pilot-sample jackknife recovers canonical rate in nested Sobol' estimation","Split jackknife nested estimator matches ordinary Monte Carlo rate","Standard nested and unbiased jackknife attain slower nested rates for Sobol'","Pick-freeze estimators as nested sims with fixed inner samples for Sobol'"],"cache_read_input_tokens":128,"weakest_assumption_plain":"The claimed rate separation rests on technical moment and growth conditions (finite moments of the conditional expectation and of the inner estimator, and suitable growth of the inner sample size with total budget) that are assumed rather than verified for general black-box simulators.","fun_headline_variants_meta":{"raw":{"variants":["Split jackknife attains O(N^{-1}) MSE rate for Sobol' under CMC","Pilot-sample jackknife recovers canonical rate in nested Sobol' estimation","Split jackknife nested estimator matches ordinary Monte Carlo rate","Standard nested and unbiased jackknife attain slower nested rates for Sobol'","Pick-freeze estimators as nested sims with fixed inner samples for Sobol'"]},"model":"grok-4.5","cost_usd":0.011032,"raw_usage":{"total_tokens":2452,"prompt_tokens":824,"num_sources_used":0,"completion_tokens":81,"cost_in_usd_ticks":110320000,"prompt_tokens_details":{"text_tokens":824,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1547,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":824,"tokens_out":81,"duration_ms":15665,"temperature":1.0,"reasoning_tokens":1547,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-08T19:46:32.536678+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Under crude Monte Carlo with total budget N, plot log-MSE versus log-N for the split jackknife (optimal allocation) against the standard nested and unbiased jackknife estimators; if the split jackknife slope is not asymptotically -1 while the others remain steeper, the central rate claim is false.","supporting_citations":[],"review_version":1}