{"id":"fa966445-7e52-4cc2-b053-6a1c3b45238e","arxiv_id":"2509.01091","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Averaging many randomly subsampled OLS control variate estimators is competitive with regularized ZVCV and much faster.","lead":"This paper proposes a new way to build control variates, a standard Monte Carlo variance reduction technique, by averaging many randomly subsampled least squares models instead of fitting one large penalized model. The authors report that the ensemble approach matches the statistical efficiency of regularized control variates while running much faster.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Unproved transfer of the LeJeune bagged-OLS/ridge equivalence to the ZVCV setting is load-bearing; the paper concedes the assumptions fail, and Experiment 1's largest-S row reverts to standard ZVCV, so the ensemble mechanism is not established.","rationale":"The reader's weakest assumption identifies the same load-bearing point: the implicit-regularization transfer from LeJeune et al. is asserted, not proved, and the paper's own text concedes the required assumptions are violated. I add two concretizations: the Portier and Segers result cited for growing J* does not cover averaging of random-subset OLS fits, and the largest-sample row of Experiment 1 reverts to standard ZVCV by the algorithm's own stopping rule, so that row is not evidence for the ensemble mechanism. These are addressable gaps rather than demonstrated failures. The method is coherent, the experiments are a reasonable first step, and the conditional verdict remains appropriate.","tokens_in":14943,"tokens_out":9032,"duration_ms":115788,"concrete_test":"Release the implementation and run a controlled benchmark: (i) a Gaussian-linear positive control satisfying LeJeune assumptions, (ii) a non-Gaussian, correlated, misspecified case with f outside the base polynomial span, and (iii) the two paper experiments, each with 100 bootstrap replicates. Compare SA25/DO25 ensemble ZVCV against r-ZVQ with 10-fold CV, reporting bias-corrected CIs for SE and OE, and sweep k in {5,25,50}, J* rules, and Qbase. If tuned ridge beats the best ensemble by more than the bootstrap margin in case (ii), the implicit-regularization transfer is refuted; if the ensemble remains within that margin, the concern is resolved.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim requires that random-subset OLS averaging supplies the benefit of regularized ZVCV without its cost. The only theoretical route offered is LeJeune et al.'s equivalence between bagged-OLS ensembles and optimally tuned ridge, but that equivalence assumes Gaussian predictors, a correctly specified linear model, and asymptotic subsampling. In the actual construction, Z is built from L2-polynomials of correlated MCMC draws, f is not in the span of the selected features, and the sample is autocorrelated. Section 3.3.2 and Section 5 explicitly concede that these assumptions are typically violated in regression-based control variates. Portier and Segers' growing-J* consistency applies to a single regression control variate, not to averaging k random-subset OLS fits with estimated weights, so it does not supply the missing transfer. Consequently, the statistical-efficiency claim is supported only by two simulations with no standard errors, no released code, and default hyperparameters chosen in unreported preliminary assessments. Moreover, in Experiment 1 at S=3000, J* >= J, so Algorithm 2 returns standard ZVCV; the largest-sample row is not an ensemble result. If the ridge-equivalence fails in the actual non-Gaussian, misspecified, dependent regime, the 'competitive but faster' conclusion is unsupported.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces 'ensemble control variates,' specifically for zero-variance control variates (ZVCV). Instead of solving one penalized regression over all J polynomial control variates, Algorithm 2 fits k independent OLS regressions on random subsets of J* < J columns of the ZVCV design matrix and averages the resulting intercepts, optionally weighting them via sample covariance (DO) or quadratic programming (MO). The motivation is the result of LeJeune et al. (2020) that bagged OLS ensembles with feature randomness approximate optimally tuned ridge regression. The authors propose hyperparameter heuristics, including a 'semi-exact' variant that always includes all monomials up to order Qbase, and compare SA/DO/MO variants with ZVCV, regularized ZVCV, CF and SECF on two ODE-based posterior inference tasks (Lotka-Volterra and Friberg-Karlson) with S in {100,300,1000,3000}, repeated 100 times. The central claim is that ensemble ZVCV 'are competitive with regularised ZVCV methods in terms of statistical efficiency, but are substantially faster.'","tokens_in":15228,"tokens_out":7630,"duration_ms":83197,"significance":"If the claim were fully substantiated, this would be a practically useful and very simple alternative to penalized ZVCV in medium-dimensional, expensive-sampling settings. The simulation models are realistic and non-toy, and the paper is clearly written and openly acknowledges the heuristic nature of its theory. The chief strengths are simplicity and the use of two challenging Bayesian models. However, as it stands the evidence is not yet at the level needed for the central claim. The paper does not make code available at review time, gives no uncertainty quantification on the efficiency estimates, and its largest-sample Experiment-1 row reduces to standard ZVCV by construction. The theoretical route via LeJeune et al. is explicitly acknowledged to be based on violated assumptions. The contribution is nevertheless promising and the presentation is honest about its limitations.","major_comments":[{"comment":"The statistical-efficiency claim relies on the transfer of LeJeune et al.'s equivalence between bagged OLS ensembles and optimally tuned ridge regression to the ZVCV setting. The manuscript itself states in §3.3.2 that 'these assumptions are typically violated in regression-based control variates (e.g., due to non-Gaussian control variates and misspecified linear models)' and repeats in §5 that 'these assumptions are generally violated in the ZVCV context and our ensemble algorithm differs somewhat.' No theorem, simulation diagnostic, or calibration experiment is provided to show that the ensemble still behaves like an optimally tuned ridge in the actual regimes (non-Gaussian correlated MCMC draws, misspecified basis, dependence). Without this bridge, the 'competitive in statistical efficiency' part of the abstract is an empirical claim supported only by two point-estimate tables. This i","section":"§3.3.2, §5"},{"comment":"In Experiment 1 at S=3000, Algorithm 2's first line returns the standard ZVCV estimate whenever J* ≥ J. For d=8 and Qmax=5, J = binom(13,5)-1 = 1286 while J* = min(0.8*3000, 25*sqrt(3000)) = 1369, so J* > J. Correspondingly, the S=3000 entries for SA25, SA50, DO25, DO50, MO25, and DO50 in Table 3 are all identical to ZV5 (43025.02). Thus the largest-sample column of Experiment 1 contains no ensemble results at all; this row cannot be used as evidence for the ensemble mechanism, and it weakens the 'competitive' claim in the large-S regime. The authors should either choose a Qmax or a J* growth rate that keeps J* < J at S=3000, or explicitly exclude this row from ensemble comparisons.","section":"§4.3.1, Algorithm 2, Table 3"},{"comment":"The simulation study reports SE and OE as point estimates from 100 replicates, with no standard errors, confidence intervals, or other uncertainty quantification. Many of the conclusions ('slightly outperform,' 'improves as k increases,' 'DO-based method generally performs worse when S is small') are based on differences of a few percent or less; without sampling variability it is impossible to judge whether these differences are real. Additionally, the default hyperparameters (c1=0.8, c2=25, k=25, Qbase) were selected in 'preliminary empirical assessments' (§3.3) that are not documented, and no sensitivity analysis is reported. The central comparison therefore rests on unquantified and partly unreported simulation choices. I would like to see, at minimum, Monte Carlo standard errors or intervals and a sensitivity check for the main defaults.","section":"§4.1, Tables 3–4"}],"minor_comments":[{"comment":"The statement 'J* ∈ O(S^{-1/2})' cannot be right for an integer growing with S. The default J* = min(c1 S, c2 sqrt(S)) is O(sqrt(S)). Please state the correct rate from Portier & Segers (2019) and align the heuristic.","section":"§3.3.2"},{"comment":"The final row is labeled 'DO50' but should presumably be 'MO50', since MO25 appears in the previous row and the DO50 row is duplicated.","section":"Table 3"},{"comment":"Line 7, 'Compute a weight vector w', is ambiguous; the weighting method (SA, DO, MO) should be an explicit input or the algorithm should be split into separate variants.","section":"Algorithm 2"},{"comment":"The statement 'Will be available shortly' makes the reproducibility claim unverifiable at review time. A permanent DOI or link should be included.","section":"Footnote 2"},{"comment":"The proposition is stated without proof. Since it follows immediately from the known zero-variance property of ZVCV under Gaussian π and polynomial f, a one-sentence proof would improve completeness.","section":"Proposition 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope for a computational statistics journal. My recommendation is major revision rather than reject because the issues are fixable: add uncertainty quantification, repair the S=3000 row, and either provide a theoretical bridge or clearly reframe the contribution as a heuristic with empirical support. The lack of code at review time is a further concern. The authors should also clarify the relationship between their ensemble definition and earlier uses of ensembles in control variates."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a sensible applied paper that proposes averaging over random subsets of polynomial Stein control variates as a cheap alternative to penalized ZVCV. The idea is simple, clearly explained, and the two ODE-based Bayesian benchmarks are well chosen. The semi-exact construction (always include low-order monomials, randomly sample higher-order ones) is genuinely new and gives the exactness guarantee in Proposition 1 for the Gaussian/polynomial case. Credit where due: the authors are explicit that the implicit-regularization analogy to bagged OLS/ridge (LeJeune et al.) is heuristic in the ZVCV setting; they flag in Sections 3.3.2 and 5 that the assumptions are typically violated. That is honest.\n\nThe soft spots are not hidden but they are real. First, no code or data artifact is shipped (\"will be available shortly\"), the reported efficiency ratios have no standard errors or confidence intervals, and the default hyperparameters (c1=0.8, c2=25, k=25, Qbase) are justified only by unreported preliminary assessments. For a methods paper whose entire claim is computational speed and statistical efficiency, that is a reproducibility gap. Second, the strongest evidence for the ensemble mechanism disappears at S=3000 in Experiment 1: with c2=25, J* = min(2400, 1369) >= J, so Algorithm 2 returns ordinary full ZVCV. The rows that look like ensemble results at S=3000 are just ZV5. Third, the theoretical bridge is genuinely unproved: LeJeune et al.'s equivalence assumes Gaussian predictors and a correctly specified linear model, while Z is built from dependent, non-Gaussian L2-polynomial features and the model is misspecified. The paper doesn't pretend otherwise, but that means the \"competitive but faster\" conclusion rests on two simulations, not on a theorem. Portier and Segers' consistency result covers a single growing-J regression CV, not k random-subset OLS fits averaged with estimated weights.\n\nIf I were refereeing, I'd ask for code, error bars on the MSE ratios, a sensitivity analysis for the default hyperparameters, and a cleaner separation of \"ensemble helps\" from \"higher-order terms help\"—for example, a comparison where the ensemble method is forced to use strictly fewer than J selected columns at every sample size. The method itself is coherent and the benchmarks are reasonable first evidence. It deserves peer review, but it needs revision before publication.","headline":"Ensemble ZVCV is a plausible, honest speed/accuracy trade-off, but the paper currently lacks code, uncertainty quantification, and its largest-sample evidence collapses back to standard ZVCV.","tokens_in":15727,"tokens_out":1962,"would_cite":true,"duration_ms":21593,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F15","65C05","62J07"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that averaging OLS fits on random polynomial subsets yields control variates competitive with regularized ZVCV but much faster.","keywords":["ensemble learning","control variates","ZVCV","variance reduction","MCMC","Bayesian inference","implicit regularization","Langevin-Stein operator"],"falsifier":"Compare ensemble ZVCV against ridge ZVCV with oracle-selected lambda on a suite of correlated, non-Gaussian ZVCV design matrices: if the ensemble's mean squared error exceeds the oracle ridge estimator's by more than its runtime advantage at any sample size, the paper's central claim is refuted.","tokens_in":14797,"feed_emoji":"📉","tokens_out":5098,"duration_ms":58155,"temperature":0.7,"pith_summary":"The paper tries to establish that ensemble learning can replace expensive penalized regression when building Stein-based control variates for Monte Carlo and MCMC estimators. Specifically, it proposes ensemble zero-variance control variates (ensemble ZVCV): instead of fitting one regularized regression over a high-dimensional polynomial basis, average many cheap ordinary least squares fits, each on a randomly selected subset of polynomial control variates. The core claim is that this averaging reproduces the variance-reduction quality of regularized ZVCV while being substantially faster. If true, Bayesian posterior means can be estimated with large variance reduction without the cross-validation and per-integrand fitting costs of ridge or LASSO ZVCV. The paper demonstrates the claim on two expensive posterior estimation problems, where the ensemble methods generally have the best overall efficiency.","feed_headline":"Bagged control variates match ridge accuracy at a fraction of the cost","feed_subtitle":"Averaging random polynomial control variates gives Stein-based variance reduction without costly cross-validated tuning.","key_machinery":"The engine is the ensemble ZVCV estimator: a weighted average of k intercepts from constrained least-squares fits, each fit using a randomly selected subset of J* columns from the ZVCV design matrix Z, with low-order base polynomial terms always included in the semi-exact version. The statistical mechanism is implicit regularization through averaging: under standard regression assumptions, a large bagged-OLS ensemble with feature randomness has the same risk as optimally tuned ridge regression, so the ensemble can obtain ridge-like variance reduction without explicitly solving a penalized regression or tuning its penalty. The paper also provides hyperparameter heuristics: set J* to grow like","core_discovery":"The central discovery is a new way to make zero-variance control variates practical in medium-to-high-dimensional settings where the polynomial basis is larger than the sample size. Standard ZVCV solves a linear regression whose design matrix is built from Langevin-Stein operator applied to polynomial terms; when the number of terms J exceeds the sample size S, the OLS problem is non-identifiable and can produce variance-inflated estimates. The paper's proposal is to randomly draw subsets of polynomial columns, fit an OLS ZVCV estimator on each subset, and average them, optionally with weights derived from simple averaging, an additional OLS step, or Markowitz-style optimization. A 'semi-exa","pith_inferences":["A likely limit of the approach is that the implicit-regularization analogy degrades when the ZVCV design columns are strongly correlated and non-Gaussian; in such regimes an oracle-tuned ridge ZVCV may still dominate, and the paper's two experiments may not reveal that gap.","A natural extension would be to replace simple random column selection with deterministic or low-discrepancy sampling, which could reduce Monte Carlo error in the ensemble estimate and allow smaller ensemble sizes k.","An analytic bias-variance decomposition of the ensemble estimator, separating the bias introduced by dropping polynomial terms from the variance reduction due to averaging, would give principled rules for choosing J* and k beyond the paper's heuristics.","The semi-exact idea suggests a general recipe for other control variate families: always include a small exactly-correct subspace and bag over random perturbations of the remaining approximation space."],"forward_implications":["Posterior mean estimation from gradient-based MCMC can get substantial variance reduction at a fraction of the runtime of regularized ZVCV, since no cross-validation for the penalty parameter is needed.","Multiple expectations can be estimated simultaneously from the same design matrix using cheap OLS fits, rather than solving a separate penalized regression for each integrand.","Practitioners no longer need to choose the polynomial order Q precisely: the semi-exact ensemble includes low-order terms exactly and samples higher-order terms, with the default configuration performing well across the studied settings.","If the implicit-regularization transfer holds broadly, ensemble ZVCV offers a nearly tuning-free alternative to ridge and LASSO ZVCV in the over-parameterized regime.","The ensemble framework is not tied to polynomial bases and could be applied to other Stein-based control variate classes, such as RKHS or neural-network parametrizations."],"supporting_citations":[{"why":"Supplies the theoretical motivation that bagged OLS ensembles with feature randomness implicitly regularize like optimally tuned ridge regression.","marker":"LeJeune et al. (2020)"},{"why":"Establishes consistency and the super-root-S convergence rate for regression-based control variates with a growing number of covariates, grounding the heuristic for choosing J*.","marker":"Portier & Segers (2019)"},{"why":"Introduces ZVCV and the zero-variance property for Gaussian targets with polynomial integrands, which the semi-exact ensemble variants build on.","marker":"Mira et al. (2013)"},{"why":"Introduces regularized ZVCV via LASSO and ridge regression, which serves as the main baseline for statistical and computational efficiency.","marker":"South et al. (2023)"},{"why":"Documents the variance inflation of ridgeless OLS in over-parameterized ZVCV and motivates control-variate selection and regularization.","marker":"Leluc et al. (2021)"},{"why":"Establishes the Langevin-Stein operator as a source of zero-mean control functionals and provides the control-functional baseline (CF).","marker":"Oates et al. (2017)"}],"fun_headline_variants":["Ensemble ZVCV: fast variance reduction without tuning","Averaged control variates match ridge at lower cost","Random subspace ensembles speed up Stein ZVCV","Bagging ZVCV: competitive accuracy, much faster","Ensemble learning trims ZVCV without accuracy loss"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is that a random-forest-style average of OLS estimators computed on random subsets of polynomial control variates reproduces the variance reduction of optimally tuned ridge regression, even though the paper states that the regression assumptions behind that equivalence are typically violated for control variates.","fun_headline_variants_meta":{"raw":{"variants":["Ensemble ZVCV: fast variance reduction without tuning","Averaged control variates match ridge at lower cost","Random subspace ensembles speed up Stein ZVCV","Bagging ZVCV: competitive accuracy, much faster","Ensemble learning trims ZVCV without accuracy loss"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00069,"raw_usage":{"total_tokens":2965,"prompt_tokens":754,"completion_tokens":2211,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":498,"completion_tokens_details":{"reasoning_tokens":2146}},"tokens_in":498,"tokens_out":2211,"duration_ms":19097,"temperature":1.0,"reasoning_tokens":2146,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T12:53:09.662737+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compare ensemble ZVCV against ridge ZVCV with oracle-selected lambda on a suite of correlated, non-Gaussian ZVCV design matrices: if the ensemble's mean squared error exceeds the oracle ridge estimator's by more than its runtime advantage at any sample size, the paper's central claim is refuted.","supporting_citations":[],"review_version":1}