{"id":"d5f4ada1-3c06-4986-9325-dcdbfbe7ecd4","arxiv_id":"1908.04904","paper_version":4,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"A distributed least squares approximation combines local estimators weighted by inverse covariance to match global estimator efficiency with one communication round.","lead":"This paper proposes DLSA, a distributed method that fits regression models on many computers and combines the local results with one round of communication, achieving the same statistical accuracy as a model fit on all data. It adds an adaptive Lasso step and a distributed BIC for variable selection, and demonstrates the method on a 52 GB airline dataset with Spark.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The √N-efficiency claim hinges on the information equality in Condition (C4); if the working loss is misspecified, DLSA's inverse-Hessian weights need not reproduce the global estimator's covariance.","rationale":"The reader's weakest_assumption is Condition (C4), and my stress-test identifies the same load-bearing point: the claimed equality between DLSA and the global estimator is not a free consequence of weighted averaging but depends on the information equality holding for every local loss. Under correct specification the theorem is plausible and the proof sketch is broadly consistent with standard M-estimator arguments; the issue is that C4 is doing the work and is not guaranteed by the method's stated scope. The numerical red flags noted by the reader (duplicate Poisson/logistic table, missing promised examples, one-round versus two-round communication) bear on finite-sample credibility but are secondary to the asymptotic claim. The Appendix A.2 inequality typo reinforces the need for careful proofreading but does not by itself undermine the theorem, because the stated condition n≫√N does imply K=o(√N). Since the central concern matches the reader's identified weakest assumption and the verdict is already CONDITIONAL, no change is needed; the conditionality should stand until the authors either restrict the efficiency claim to correctly specified likelihoods or provide a misspecification analysis.","tokens_in":32449,"tokens_out":9498,"duration_ms":104374,"concrete_test":"Run a misspecification simulation: generate overdispersed count data, e.g., Y_i from a Negative Binomial with mean exp(X_i^T θ0) and dispersion φ=2, but fit a Poisson working likelihood. Use N=100,000, K=20, p=8, θ0 as in Example 3, and 500 replications. Compare DLSA θ̃ with the global Poisson MLE θ̂ in RMSE per coefficient and 95% coverage. If DLSA RMSE is systematically above the global estimator's for nonzero coefficients, or if DLSA coverage departs materially from nominal, the efficiency claim fails outside Condition (C4). As a control, also combine the same local estimators with inverse-sandwich weights using local J_k^{-1} I_k J_k^{-1}; if that restores efficiency, the weighting scheme is the cause.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is Theorem 1: √N(θ̃−θ0)→d N(0,Σ) with Σ=(Σ_k α_k Ω_k)^{-1}, the same covariance as the global minimizer θ̂. This equality requires two separate facts: (i) each local estimator has covariance Σ_k=Ω_k^{-1}, and (ii) the global estimator has covariance (Σ_k α_k Ω_k)^{-1}. Both are delivered by the same assumption, Condition (C4), which asserts that the score outer product equals the expected Hessian for every local worker. Condition (C4) is an information-equality assumption, not merely a regularity condition. It holds automatically when L is a correctly specified log-likelihood with canonical link, but the paper presents DLSA as applying to 'a large family of regression problems' and does not list correct specification among its assumptions. If the working loss is misspecified, √n_k(θ̂_k−θ0) has a sandwich asymptotic covariance J_k^{-1} I_k J_k^{-1}, while the global estimator has covariance J^{-1} I J^{-1}; DLSA weights only by J_k, so the claimed equality to the global estimator's distribution does not follow. A separate typo appears in Appendix A.2, where 'n≫√N' is said to imply 'K≫√N'; the required condition is K=o(√N), which does follow from n≫√N. That typo is emendable, but the C4 dependence is the load-bearing issue.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops a one-shot distributed estimation procedure, DLSA, in which each worker fits the same parametric model locally and the master combines the local estimators by inverse-variance-type weights. Under Conditions C1–C5 it claims that the combined estimator has the same √N-limiting distribution as the global estimator, and it adds an adaptive-Lasso version with a new DBIC criterion that is claimed to have oracle and selection-consistency properties. The paper reports simulations for linear, logistic, and Poisson regressions under i.i.d. and heterogeneous covariate distributions, and it illustrates the method on a 52 GB airline dataset using a Spark implementation.","tokens_in":32676,"tokens_out":16917,"duration_ms":162354,"significance":"If the theoretical claims are correct, the paper provides a communication-efficient and practically useful alternative to iterative distributed estimation, and it goes beyond much of the one-shot literature by explicitly addressing heterogeneous worker-specific covariate distributions and by providing a tuning-parameter selection criterion. The proofs are self-contained from explicit conditions, and the availability of an open-source Spark implementation is a concrete strength. However, the numerical validation currently contains a duplicated table, and the scaling conventions for the covariance matrix are inconsistent between the main text, the proofs, and the reported coverage probabilities. These issues must be resolved before the claims can be accepted.","major_comments":[{"comment":"Table 3, labeled Poisson regression (Example 3), is numerically identical to Table 2, labeled logistic regression, in both panels (N = 20, K = 10 and N = 100, K = 20). The paper therefore reports no actual results for the Poisson example, and the sentence in Section 4.4 discussing the Poisson model refers to a table that is a copy of the logistic results. This must be corrected by rerunning the simulation and by checking all tables for similar copy-and-paste errors.","section":"Section 4.4, Table 3"},{"comment":"Condition (C4) is described as a local convexity condition, but it is the information equality E[score score^T] = E[Hessian] for each worker. Under a misspecified working loss, the local estimator has the sandwich covariance J_k^{-1} I_k J_k^{-1}, and the inverse-Hessian weights in (2.2) do not generally reproduce the global estimator's limiting distribution. Please state explicitly that L is a correctly specified negative log-likelihood, or provide a misspecification analysis; the unqualified claim that DLSA is statistically as efficient as the global estimator is otherwise broader than the proof supports.","section":"Section 2.2, Condition (C4)"},{"comment":"The DBIC as printed, DBIC_lambda = (theta_tilde_lambda - theta_tilde)^T Sigma_hat^{-1} (theta_tilde_lambda - theta_tilde) + log N * df_lambda, cannot be consistent with Theorem 5. With Sigma_hat denoting the asymptotic covariance matrix of sqrt(N)(theta_tilde - theta_0) from (2.2) and Proposition 1, the quadratic term is O_p(1/N), while the penalty diverges, so the criterion would select the null model. Appendix B.3 uses df_lambda (log N)/N, so the main-text formula appears to be missing a factor 1/N. Please correct (3.4) and use one unambiguous convention for Sigma_hat throughout.","section":"Section 3.2, Eq. (3.4)"},{"comment":"The confidence intervals are defined with the standard error equal to a diagonal element of Sigma_hat = (sum_k alpha_k Sigma_hat_k^{-1})^{-1}. With Sigma_hat as defined in (2.2) and Proposition 1, this is the asymptotic covariance matrix of sqrt(N)(theta_tilde - theta_0), so the interval width is O(1) and coverage would be essentially one, not approximately 0.95 as reported in Tables 1-3. The intervals should use Sigma_hat/N, or Sigma_hat should be redefined as the covariance of theta_tilde itself. Please clarify and reconcile the stated formula with the reported coverage probabilities.","section":"Section 4.3, CI formula"}],"minor_comments":[{"comment":"The sentence 'by the condition n >> sqrt(N), we have K >> sqrt(N)' reverses the implication: since n = N/K, n >> sqrt(N) implies K << sqrt(N), which is exactly K = o(sqrt(N)).","section":"Appendix A.2"},{"comment":"The proofs of Theorems 1 and 2 cite Condition (C6), which is introduced in Section 3.1 and is not assumed by these theorems; the variance calculation should cite Condition (C4).","section":"Appendix A.2 and A.3"},{"comment":"The phrase 'asymptotic covariance estimate' for Sigma_hat_k is ambiguous: it should be stated whether this is the covariance of sqrt(n_k)(theta_hat_k - theta_0) or of theta_hat_k itself; the same distinction underlies the factor-N issue in the DBIC and confidence interval formulas.","section":"Section 2.2"},{"comment":"The abstract lists Cox's model as a target example, but no Cox simulation or verification that the Cox partial likelihood satisfies Condition (C4) is provided; a remark or additional example would be helpful.","section":"Abstract and Section 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is likely salvageable, but the duplicated Table 3 and the scaling inconsistency in the DBIC and confidence interval formulas need to be fixed before the paper can be recommended for publication. I would ask for a careful audit of all simulation tables and of every formula involving Sigma_hat. The theoretical results appear plausible under correct specification, and the Spark demonstration is a strength."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea isn't new — inverse-variance weighting of local estimators is textbook meta-analysis, and the quadratic approximation comes straight from Wang and Leng (2007) — but the paper does a solid job extending it to heterogeneous workers, adding a sensible two-step bias correction and a distributed BIC. The proofs are self-contained and the conditions are explicit. That’s real work, and the Spark implementation on a 52 GB airline dataset is a useful practical contribution.\n\nNow the problems, in proportion. The big one: Table 3, labeled Poisson regression, is numerically identical to Table 2, the logistic regression table. I checked the numbers column by column. This is not a small typo; it destroys confidence in the finite-sample evidence. The abstract promises five simulation examples but only three appear. The one-round communication claim is true for DLSA but TWLSE takes two rounds, and the paper doesn’t qualify that. These need to be fixed before anything else.\n\nThe theoretical side is cleaner. The efficiency claim hinges on Condition (C4), which is an information equality: expected outer product of the score equals expected Hessian. That holds for a correctly specified likelihood, and the paper is effectively doing MLE, but it never says so explicitly. If the working loss is misspecified, the sandwich covariance differs and inverse-Hessian weights don’t reproduce the global estimator’s distribution. That’s a limitation worth stating rather than a hidden flaw, but the paper should own it. There’s also a minor typo in Appendix A.2: n≫√N implies K≪√N, not K≫√N.\n\nThe citation pattern is honest. They build on Wang and Leng, Jordan et al., Zhang et al., and the heterogeneity point is a genuine contribution to the one-shot literature. They also ship code and a data analysis, which I count as credit.\n\nWho’s this for? People working on divide-and-conquer regression and communication-efficient estimation, and practitioners with large data and strong workers. It deserves a serious referee — the theory is plausible and the problem is real — but the referee should demand a corrected numerical section, explicit discussion of the correct-specification assumption, and a fix for the missing examples. I’d engage with it after those revisions, not before.","headline":"The theory is worth engaging, but the numerical section currently fails a basic integrity check.","tokens_in":33227,"tokens_out":2695,"would_cite":false,"duration_ms":32270,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F12","62J07"],"pacs":[],"model":"deepseek-v4-flash","headline":"A one-round, inverse-covariance weighted average of local regression fits achieves the same asymptotic efficiency as the full-data estimator.","keywords":["distributed systems","least squares approximation","one-shot communication","inverse covariance weighting","adaptive Lasso","oracle property","distributed BIC","heterogeneous data"],"falsifier":"Simulate a Poisson log-linear working model on negative-binomial data distributed across workers, with each worker's sample size $n$ much larger than $\\sqrt{N}$, and compare the DLSA estimator's sampling covariance with that of the global estimator; if they differ in any component at the claimed rate, the information-equality condition underlying the theorem is violated.","tokens_in":32187,"feed_emoji":"📡","tokens_out":5601,"duration_ms":56333,"temperature":0.7,"pith_summary":"The paper proposes a distributed estimation method for regression problems on a master-and-worker system: each worker fits the model locally and sends its estimate and an estimated covariance to the master, which combines them by inverse-covariance weighting. The central claim is that this one-round combination is asymptotically as efficient as the global estimator, meaning it has the same limiting normal distribution with the same covariance. The paper further shows that an adaptive Lasso step on the master, guided by a new distributed Bayesian information criterion, selects the true model consistently and achieves the oracle property. This matters because it would give large-scale regression analysis the statistical benefits of full-data estimation while avoiding the communication cost of iterative distributed algorithms.","feed_headline":"One communication round matches full-data estimation","feed_subtitle":"Inverse-covariance weighting of local fits recovers the global estimator's asymptotics and consistent model selection.","key_machinery":"The load-bearing object is the local quadratic approximation of the global loss: expanding each local loss around its local minimizer and dropping the linear term gives a weighted least squares objective $\\tilde{L}(\\theta)=\\sum_k \\alpha_k (\\theta-\\hat{\\theta}_k)^\\top \\hat{\\Sigma}_k^{-1}(\\theta-\\hat{\\theta}_k)$, whose minimizer is an explicit inverse-covariance weighted average. The inverse covariance $\\hat{\\Sigma}_k^{-1}$ is the key ingredient: it lets the master combine heterogeneous local fits by down-weighting workers with larger asymptotic variance. The same quadratic form, evaluated on the master, serves as the loss for adaptive Lasso shrinkage and as the basis for the distributed BIC selection criterion.","core_discovery":"The central result is Theorem 1: under regularity conditions including each local sample size $n$ growing faster than $\\sqrt{N}$, the DLSA estimator $\\tilde{\\theta} = (\\sum_k \\alpha_k \\hat{\\Sigma}_k^{-1})^{-1} \\sum_k \\alpha_k \\hat{\\Sigma}_k^{-1} \\hat{\\theta}_k$ satisfies $\\sqrt{N}(\\tilde{\\theta}-\\theta_0)\\to_d N(0,\\Sigma)$, the same asymptotic distribution as the global estimator. Intuitively, the inverse-covariance weighting turns a collection of $\\sqrt{n}$-consistent local estimators into a $\\sqrt{N}$-consistent global one, and its bias term vanishes when workers are large enough. A two-step variant improves the bias rate and relaxes the local-sample condition to $n\\gg N^{1/4}$, while iterating the procedure $m$ times with $m/\\log N\\to\\infty$ allows even $n=O(1)$. The paper also proves that the adaptive Lasso on the master is selection consistent and oracle-efficient, and that the proposed distributed BIC consistently identifies the true model.","pith_inferences":["The inverse-covariance weighting principle is more general than the paper's likelihood setting: any collection of $\\sqrt{n}$-consistent, independently computed estimates with reported covariance matrices could be combined the same way, though the efficiency guarantee would require the reported covariance to be the true one.","Because the efficiency theorem rests on the information equality in Condition (C4), a practical check would be to compare each worker's score outer product with its Hessian; a large discrepancy would signal that the DLSA estimator may no longer match the global estimator's covariance.","The paper's appendix suggests that bootstrap covariance estimates could replace the Hessian when it is hard to compute, which points toward a plug-in variant of DLSA for semi-parametric or robust losses.","The DBIC construction could plausibly be adapted to other penalties (for example, SCAD or MCP) because it only requires a consistent global quadratic surrogate and a consistent reference estimator."],"forward_implications":["Large regression problems such as linear, logistic, Poisson, and Cox-type models can be analyzed on a distributed cluster with a single round of communication while retaining full-data asymptotic efficiency.","Heterogeneous covariate distributions across workers do not destroy efficiency, because per-worker inverse-covariance weighting automatically accounts for differing local variability.","Variable selection and tuning-parameter selection can be done entirely on the master with negligible extra communication, and the selected model is consistent.","The two-step and multi-step variants extend the approach to settings with many weak workers or severe local memory constraints.","On very large datasets, the method requires modest worker memory and is substantially faster than conventional distributed algorithms, as demonstrated by the 52 GB airline application."],"supporting_citations":[{"why":"Supplies the least squares approximation idea and the unified Lasso formulation that DLSA builds on.","marker":"Wang and Leng (2007)"},{"why":"Provides the adaptive Lasso penalty with oracle properties that the shrinkage step adopts.","marker":"Zou (2006)"},{"why":"Gives the asymptotic normality of local M-estimators that each worker's estimator relies on.","marker":"Lehmann and Casella (2006)"},{"why":"Defines the one-shot distributed estimation baseline against which DLSA is compared.","marker":"Zhang et al. (2013)"},{"why":"Provides the iterative CSL method whose local-Hessian averaging is compared with DLSA and is shown to be less efficient under heterogeneity.","marker":"Jordan et al. (2019)"},{"why":"Supplies the nonconcave penalized likelihood oracle-property framework and regularity conditions used in the proofs.","marker":"Fan and Li (2001)"},{"why":"Provides the LARS algorithm used to compute the adaptive Lasso solution path on the master.","marker":"Efron et al. (2004)"},{"why":"Gives the pre-feature screening procedure that reduces the feature dimension before the DLSA shrinkage step.","marker":"Li et al. (2020)"}],"fun_headline_variants":["One communication round matches full-data estimation","Single-shot distributed regression matches global estimator","One-step weighting achieves full-data efficiency","Distributed least squares with oracle properties in one round","One-pass distributed estimation matches ideal asymptotics"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The defining efficiency result assumes that within each worker the information equality holds: the variance of the local score equals the expected curvature of the local loss, so the inverse covariance is the right weight.","fun_headline_variants_meta":{"raw":{"variants":["One communication round matches full-data estimation","Single-shot distributed regression matches global estimator","One-step weighting achieves full-data efficiency","Distributed least squares with oracle properties in one round","One-pass distributed estimation matches ideal asymptotics"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000156,"raw_usage":{"total_tokens":1239,"prompt_tokens":984,"completion_tokens":255,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":600,"completion_tokens_details":{"reasoning_tokens":190}},"tokens_in":600,"tokens_out":255,"duration_ms":3346,"temperature":1.0,"reasoning_tokens":190,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:29:41.834032+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate a Poisson log-linear working model on negative-binomial data distributed across workers, with each worker's sample size $n$ much larger than $\\sqrt{N}$, and compare the DLSA estimator's sampling covariance with that of the global estimator; if they differ in any component at the claimed rate, the information-equality condition underlying the theorem is violated.","supporting_citations":[{"cited_title":"(2006), The adaptive lasso and its oracle properties, Journal of the American Statistical Association, 101, 1418--1429","cited_arxiv_id":null,"evidence_quote":"Provides the adaptive Lasso penalty with oracle properties that the shrinkage step adopts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Gives the asymptotic normality of local M-estimators that each worker's estimator relies on."},{"cited_title":"C., and Wainwright, M","cited_arxiv_id":null,"evidence_quote":"Defines the one-shot distributed estimation baseline against which DLSA is compared."},{"cited_title":"(2004), Least angle regression, Annals of Statistics, 32, 407--499","cited_arxiv_id":null,"evidence_quote":"Provides the LARS algorithm used to compute the adaptive Lasso solution path on the master."}],"review_version":1}