{"id":"c2f69486-11f5-42e4-96fe-be1f59018353","arxiv_id":"2601.22378","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"In exponential families where the target parameters equal the mean parameters, optimal control variate estimators and maximum likelihood estimators have identical asymptotic variance, giving a fixed-point algorithm for the MLE.","lead":"This paper proves that in an exponential family, an optimal control variate estimator using known sufficient statistics has the same asymptotic variance as the maximum likelihood estimator. The proof yields a fixed-point algorithm (CV-EM) that empirically converges faster and more stably than Newton-Raphson for the bivariate Normal case.","discovery_kind":"unification","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 4's variance equivalence is credible, but the advertised CV-EM convergence to the MLE is not proven: multiple fixed points are possible at small k, and the Section 6 trace example is a biased estimator, not an MLE.","rationale":"The reader's weakest_assumption identifies the mu_i = nu_i condition in Theorem 4. I agree that this condition is restrictive and is openly violated by the Section 6 trace example. However, the single most load-bearing weakness is different: even granting Theorem 4 in full, the paper's advertised algorithmic result does not follow. Theorem 4 is an asymptotic variance equality; it does not by itself imply that the fixed-point iteration in Algorithm 1 converges, or that a converged fixed point is the global MLE rather than a spurious stationary point of the score equation. The paper's own Appendix B.2 acknowledges multiple roots for the bivariate Normal cubic at small k, and no convergence proof is supplied. The abstract and introduction claim an EM algorithm converging to the MLE, while the only fully supported contribution is the variance equivalence. Section 6 strengthens the concern: it is framed as finding the MLE from known CV weights, yet the resulting estimator is admitted to be biased and the theorem's conditions do not hold. A conditional verdict is therefore the right level: the core theoretical theorem appears sound, but the algorithm claims and the trace-section interpretation need either a proof, a contraction analysis, or a restatement. My proposed multinomial experiment would directly test whether CV-EM converges to the MLE outside the bivariate Normal case, and whether non-MLE fixed points are a practical failure mode.","tokens_in":36633,"tokens_out":19390,"duration_ms":161767,"concrete_test":"Implement CV-EM for a non-Normal exponential family satisfying mu = nu, e.g. a 3-category multinomial with known cell probabilities for categories 2 and 3, estimating nu_1 only. For each of 10^4 simulated datasets at n=10 and n=50, initialize at y_1 and iterate Algorithm 1; compare the limiting value with the global maximizer of the log-likelihood found by dense grid search over the valid parameter interval. Record the proportion of runs that converge to a non-MLE root, diverge, or cycle. Additionally, evaluate the Jacobian of the fixed-point map at the true parameter; if any eigenvalue exceeds 1 in modulus, local convergence is not guaranteed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The theoretical variance equality in Theorem 4 appears to go through under the stated mu_i = nu_i condition. The load-bearing gap is the algorithmic consequence. The abstract and Section 1.2 (Contribution Two) claim Algorithm 1 (CV-EM) is an EM algorithm converging to the MLE, and Section 3 says Theorem 4 implies the CV weights give an EM algorithm. What is actually shown in Appendix B.1 is algebra: a score equation can be rearranged into a fixed point of the form nu_s = y_s + sum_{j>t} c^{(s)}_j (y_j - mu_j) when the matrix A_{si} = partial eta_i / partial nu_s is invertible. This does not establish convergence of the Gauss-Seidel iteration, nor that a fixed point is the MLE rather than another stationary point. In the bivariate Normal example the score equation is a cubic; Appendix B.2 concedes the cubic can have three real roots for small k (Table 1 shows 0.26% at k=10) and that wrong-root convergence is possible. No contraction, monotonicity, or global-convergence argument is given; Appendix B.2 explicitly leaves the rate as an open problem and says convergence depends on the type of exponential family. The only experiments are the zero-mean bivariate Normal. Section 6's Hutchinson trace example is presented as finding the MLE in the abstract, but the paper itself states the conditions of Theorem 4 are not satisfied and the closed-form fixed point is Bekas' estimator, which the text acknowledges is biased. Thus the central claim that the method leads to finding the MLE when the CV weights are known is unsupported; the variance equivalence alone does not carry the algorithm claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies the relationship between maximum likelihood estimation (MLE) and control variate estimation (CVE) in exponential families. The main theoretical result, Theorem 4, states that when the mean parameters μi coincide with the target parameters νi, the asymptotic variance of the MLE of a linear combination of the estimated parameters equals the variance of the optimal CVE that uses the remaining sufficient statistics as control variates. The authors then propose Algorithm 1 (CV-EM), a fixed-point iteration derived from the score equations, and claim it is an Expectation-Maximization algorithm converging to the MLE. They use this to discuss reproducibility of MLE/CVE-based sketching algorithms and, in Section 6, to derive an estimator for the matrix trace. Experiments on feature hashing and random projections, both based on a zero-mean bivariate Normal model, compare the fixed-point iteration with Newton-Raphson, secant, and two CVE variants.","tokens_in":36902,"tokens_out":10465,"duration_ms":90528,"significance":"If Theorem 4 is taken as proven, the paper establishes a clean variance-equivalence identity between two estimation paradigms; the proof is self-contained and the algebraic steps are consistent with the stated condition μi=νi. The experimental section is detailed and suggests that the fixed-point iteration can be numerically competitive. However, the paper's central advertised consequences, namely that Algorithm 1 converges to the MLE and that the method can find MLEs from known CV weights, are not established by the arguments provided. The trace example in Section 6 explicitly violates the theorem's conditions and the derived estimator is acknowledged to be biased, so it does not support the 'finding the MLE' claim. The contribution is therefore significant in its core variance result but requires major revision to separate proven facts from heuristic claims.","major_comments":[{"comment":"The paper claims in the abstract and in Section 1.2 that Algorithm 1 (CV-EM) is an EM algorithm converging to the MLE. Appendix B.1 only derives a fixed-point form of the score equation (Eq. 26); it does not prove that the Gauss-Seidel iteration converges, nor that a fixed point is the MLE rather than another stationary point. Appendix B.2 explicitly leaves the convergence rate as an open problem and acknowledges possible wrong-root convergence, with Table 1 reporting 0.26% of cubics having three real roots at k=10. Without a contraction, monotonicity, or global-convergence argument, the assertion that the algorithm 'converges to the MLE' is unsupported and load-bearing for the paper's practical contributions.","section":"Section 3, Appendix B.1-B.2"},{"comment":"The section is advertised as 'Finding the MLE via CVE weights' and the abstract claims the algorithm leads to finding the MLE when CV weights are known. However, the paper states that the conditions of Theorem 4 are not satisfied for Hutchinson's estimator, and the closed-form fixed point is Bekas's diagonal estimator (Bekas et al., 2007), which the text acknowledges is biased. No likelihood for tr(M) is analyzed, and the sentence 'the MLE would be a biased estimator' is not a derivation of an MLE. Thus the example does not substantiate Contribution Three.","section":"Section 6, Appendix D"},{"comment":"The paper claims that after convergence of Algorithm 1, the estimator has 'the variance of the MLE at that specific number of observations (or sketch size) k.' Theorem 4 is an asymptotic statement, proved via asymptotic normality of the MLE and the Fisher information. No finite-sample variance result for the MLE is proved, so the finite-k variance claim in Section 4 is not justified by the theorem. The experiments at small k compare MSE, but the theoretical link to the MLE variance at finite k is missing.","section":"Section 4, Observation 3"}],"minor_comments":[{"comment":"The term 'EM algorithm' is used for what is actually a score-equation fixed-point iteration. The derivation in Appendix B.1 does not involve latent variables or a standard E-step; the authors should either justify the terminology or rename it, for example as 'CV fixed-point iteration'.","section":"Section 3, Algorithm 1"},{"comment":"The step 'Using the differential relationship in Theorem 3 and the fact that μi=νi' to obtain V_n ∂η(νE)/∂νE = (1/n)[I_t;0] is terse. Please spell out the chain-rule calculation, since this is the point where the theorem's key condition enters.","section":"Appendix B, proof of Theorem 4"},{"comment":"The sentence 'the MLE would be a biased estimator' is confusing because the estimator derived is Bekas's estimator, not a proven MLE. Rephrase to state clearly that the fixed-point limit is a known biased estimator and is not claimed to be an MLE.","section":"Section 6"},{"comment":"The abstract says the algorithm is 'faster and numerically stable' without mentioning that the experiments cover only the zero-mean bivariate Normal distribution. Add the caveat to avoid overclaiming generality.","section":"Abstract, Section 5"},{"comment":"The text says 'where n/(σ11σ22-σ12^2) is factored out' but the factoring is not displayed in the equation. Showing the intermediate step would improve readability.","section":"Appendix A, Eq. (8)"},{"comment":"The notation V_n and V is used inconsistently: Theorem 2 uses V_n for Cov(y) and also defines V, while Theorem 4 partitions V as the matrix of ∂μ/∂η. Define these once and use them consistently throughout.","section":"Theorem 2 and Theorem 4"},{"comment":"The title contains 'F or' (with a space), which appears to be a typo for 'For'.","section":"Title"}],"recommendation":"major_revision","confidential_remarks":"The strongest contribution is Theorem 4's variance identity, which appears correct under its stated condition. The advertised algorithmic consequences are not established: the convergence of Algorithm 1 to the MLE is unproven, and the trace example in Section 6 explicitly violates the theorem's conditions and yields a biased estimator. I recommend a major revision in which the paper is reframed around the variance-equivalence theorem, with Algorithm 1 presented as a heuristic fixed-point iteration whose convergence remains an open problem. The 'EM algorithm' terminology and the 'finding the MLE' claims should be removed or substantially qualified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The one thing worth knowing about this paper: the theoretical core, Theorem 4, is solid and genuinely new. The paper proves that under a stated condition (mu_i = nu_i), the asymptotic variance of the MLE for a linear combination of the estimated parameters equals the variance of the optimal control variate estimator using the known components as CVs. The proof via Schur complements is clean and correct, and the paper is honest that the condition is restrictive. This is a real contribution to the sketching/estimation literature.\n\nWhat the paper does well beyond Theorem 4: it carefully distinguishes its result from Glynn and Szechtman's nonparametric equivalence and from information-geometry hints, and it draws the connection to a fixed-point algorithm (CV-EM) that is empirically fast and stable for the bivariate Normal case. The reproducibility discussion in Section 5 is useful, and the limitations section is candid.\n\nThe soft spots are where the paper overclaims. The abstract and Section 1.2 say Algorithm 1 is an EM algorithm converging to the MLE. What is actually shown is that the score equation can be rearranged into a fixed-point form; no convergence proof is given. The appendix even concedes multiple fixed points can exist for small k and leaves the rate as an open problem. So the algorithmic claim is not supported. The Hutchinson trace example in Section 6 is presented as \"finding the MLE,\" but the paper itself says Theorem 4's conditions do not hold there, and the resulting estimator is Bekas' estimator, which is biased. The abstract's phrasing is misleading. Experiments cover only the zero-mean bivariate Normal, and no code or data are released, though for a theory paper that is less critical.\n\nNone of this sinks Theorem 4. The theoretical contribution stands. But the paper needs restructuring: state the variance equivalence as the headline, present CV-EM as a heuristic fixed-point method with empirical support, and remove or explicitly qualify the convergence and MLE claims. Section 6 should be framed as a heuristic derivation of a known estimator, not as an MLE.\n\nWho is this for: people working on sketching, control variates, or exponential-family estimation. It deserves a serious referee; the theory is important enough to engage with. I would support sending it to peer review, with the expectation of major revision on the algorithm and trace sections.","headline":"Theorem 4 is a genuinely useful variance equivalence in exponential families, but the paper oversells the CV-EM algorithm as converging to the MLE and the trace example produces a biased estimator, not an MLE.","tokens_in":37510,"tokens_out":1380,"would_cite":true,"duration_ms":14866,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F10","62E20","65C05"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proves that, inside an exponential family, an optimally corrected control variate estimator and the maximum likelihood estimator have identical asymptotic variance reduction, and turns that identity into a fixed-point algorithm…","keywords":["exponential families","maximum likelihood estimation","control variates","sketching algorithms","fixed point iteration","EM algorithm","variance reduction","feature hashing"],"falsifier":"Choose an exponential family with known parameters ν_K and target ν_E where at least one μ_i ≠ ν_i, as in estimating a nonlinear function of the sufficient-statistic means; compute the MLE asymptotic variance from the inverse Fisher information and the optimal CVE variance from the Schur complement, and check whether they differ. A simpler analogue: run Algorithm 1 on the chi-squared trace-estimation setting of Section 6 and verify that the fixed point, if it converges, has variance different from the claimed MLE-variance identity.","tokens_in":36394,"feed_emoji":"","tokens_out":7573,"duration_ms":64961,"temperature":0.7,"pith_summary":"The paper sets out to prove that, in exponential families, maximum likelihood estimation and control variate estimation are asymptotically the same operation: if the parameters to be estimated equal the expected sufficient statistics, the asymptotic variance reduction of the MLE is exactly the variance reduction of an optimal CVE. That equality makes it possible to compute the MLE by iterating control-variate corrections instead of solving score equations, an EM-style fixed point procedure the paper calls CV-EM. The authors argue this removes two practical obstacles to using CVEs in sketching algorithms: theoretical variance calculations invalidated when unknown quantities are plugged into the control weights, and reproducibility failures caused by different root-finding or weight-estimation choices. On the bivariate Normal examples of feature hashing and random projections, CV-EM is shown to be numerically stable and fast, matching the best root-finding methods while avoiding Newton-Raphson's small-sample outliers.","feed_headline":"Control variates reach MLE variance in exponential families","feed_subtitle":"A fixed-point scheme from optimal control variate corrections converges to the MLE, stabilizing sketching estimates.","key_machinery":"The load-bearing object is the pair of Jacobian matrices of the exponential family: V_{ij} = ∂μ_i/∂η_j, which is also the scaled covariance of the sufficient statistics, and its inverse ($V^{{-1}}$)_{ij} = ∂η_i/∂μ_j, which exists because η and μ are in one-to-one relation via dμ = V dη. The MLE variance uses n times the upper-left block of $V^{{-1}}$; the optimal CVE variance uses the Schur complement A - $BD^{{-1}}$B^T of the same block decomposition of V. The condition μ_i = ν_i makes the Fisher information reduce to that upper-left block, and the Schur complement identity then identifies the two variance expressions. This duality is the mechanism that turns control-variate corrections into an EM-style fixed point algorithm for the MLE.","core_discovery":"On the paper's own terms, the central discovery is Theorem 4: for observations from an exponential family, with parameters split into an estimated block ν_E and a known block ν_K, if every mean parameter μ_i = ∂ψ/∂η_i = E(y_i) equals the corresponding ν_i, then the asymptotic variance of the MLE of any linear combination of ν_E equals the variance of the CVE that uses the sufficient statistics y_j from the known block as control variates. The proof computes the Fisher information of the MLE via the Jacobian ∂η/∂μ and the variance of the optimal CVE via the Schur complement of the covariance block matrix, then shows the two expressions are the same block-matrix identity when μ_i = ν_i. The theorem is used to derive Algorithm 1, a fixed-point iteration for the MLE whose updates are optimal control-variate corrections, and to argue that iterating CV weights converges to the MLE. It also gives a heuristic recovery of the diagonal estimator for Hutchinson's trace problem from a closed-form fixed point.","pith_inferences":["If the equivalence is taken as a design principle rather than just a theorem, it suggests deriving MLEs in new sketching settings by first computing expectation and covariance identities for sufficient statistics and then iterating the control-variate corrections, avoiding score-function algebra.","The condition μ_i = ν_i is narrow; the paper's own chi-squared trace example falls outside it, so the general claim should be read as an equivalence for the mean-parameter block rather than for all exponential-family parameterizations.","A testable extension would be to run CV-EM on multinomial frequency estimators under the known-marginal condition, where the equivalence should also hold, and to compare update counts with the same root-finding baselines."],"forward_implications":["CV-EM converges to the MLE by repeatedly applying optimal control-variate corrections, so one can obtain MLE-level variance without solving score equations.","Under the theorem's condition, the plug-in theoretical variance of a CVE is no longer invalid: after convergence its variance is the MLE variance at that sketch size.","Standardizing on CV-EM removes implementation-dependent discrepancies in MLE/CVE comparisons, since different root-finding methods give different small-sample behavior.","For Hutchinson's trace estimator, the fixed-point heuristic produces the diagonal estimator with variance expression 2(||M||_F^2 - Σm_ii^2)/k, giving a variance analysis that the original estimator lacked."],"supporting_citations":[{"why":"Supplies the control variate estimator construction and the optimal variance reduction formula used on the CVE side.","marker":"Lavenberg and Welch (1981)"},{"why":"Gives the MLE asymptotic normality and efficiency that define the MLE variance side.","marker":"Lehmann (1999)"},{"why":"Provides the exponential-family mean, covariance, and invertible eta-mu map used throughout the proof.","marker":"Efron (2022)"},{"why":"Supplies the Schur complement identity that equates the inverse Fisher information block with the CVE variance reduction.","marker":"Golub and Van Loan (2013)"},{"why":"Provides the random projection MLE cubic-root problem whose experimental comparison CV-EM extends.","marker":"Li et al. (2006)"},{"why":"Provides the feature hashing MLE/CVE formulation that the experiments and reproducibility discussion are based on.","marker":"Verma et al. (2022)"},{"why":"Gives the Hutchinson-estimator control variate and optimal coefficient c that the fixed-point heuristic re-derives.","marker":"Adams et al. (2018)"},{"why":"Defines the trace estimator whose diagonal fixed-point version is analyzed in Section 6.","marker":"Hutchinson (1989)"}],"fun_headline_variants":["MLE variance equals optimal control variates in exponential families","Fixed-point algorithm unifies MLE and control variates","Exponential family: MLE and control variates match asymptotically","Control variates achieve MLE variance asymptotically","New fixed-point scheme ties MLE to control variates"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument depends on each parameter being estimated being numerically equal to the expected value of the corresponding sufficient statistic on that block; when this equality fails, the MLE and CVE variance expressions no longer coincide and the main theorem does not apply.","fun_headline_variants_meta":{"raw":{"variants":["MLE variance equals optimal control variates in exponential families","Fixed-point algorithm unifies MLE and control variates","Exponential family: MLE and control variates match asymptotically","Control variates achieve MLE variance asymptotically","New fixed-point scheme ties MLE to control variates"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000479,"raw_usage":{"total_tokens":2343,"prompt_tokens":885,"completion_tokens":1458,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":501,"completion_tokens_details":{"reasoning_tokens":1377}},"tokens_in":501,"tokens_out":1458,"duration_ms":8315,"temperature":1.0,"reasoning_tokens":1377,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:37:36.310546+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Choose an exponential family with known parameters ν_K and target ν_E where at least one μ_i ≠ ν_i, as in estimating a nonlinear function of the sufficient-statistic means; compute the MLE asymptotic variance from the inverse Fisher information and the optimal CVE variance from the Schur complement, and check whether they differ. A simpler analogue: run Algorithm 1 on the chi-squared trace-estimation setting of Section 6 and verify that the fixed point, if it converges, has variance different from the claimed MLE-variance identity.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the control variate estimator construction and the optimal variance reduction formula used on the CVE side."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the MLE asymptotic normality and efficiency that define the MLE variance side."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the exponential-family mean, covariance, and invertible eta-mu map used throughout the proof."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Schur complement identity that equates the inverse Fisher information block with the CVE variance reduction."},{"cited_title":"D., Pratap, R., and Thakur, M","cited_arxiv_id":null,"evidence_quote":"Provides the feature hashing MLE/CVE formulation that the experiments and reproducibility discussion are based on."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the trace estimator whose diagonal fixed-point version is analyzed in Section 6."}],"review_version":1}