Pith. sign in

REVIEW 3 major objections 5 minor 73 references

High-Dimensional Model Averaging via Cross-Validation

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Cross-validation weight selection makes high-dimensional model averaging asymptotically optimal, both when every candidate model is misspecified and when a correct model is present.

desk verdict Genuinely novel and worth refereeing, but the nonconvex-penalty theory and practice gap needs closing before the empirical SCAD/MCP claims are supported. read the letter →

arxiv 2506.08451 v1 pith:Y2NIQUAF submitted 2025-06-10 math.ST stat.TH

Zhengyan Wan , Fang Fang , Binyan Jiang This is my paper · ORCID
classification math.STstat.TH MSC 62J0762F1262F40
keywords modelaveragingcross-validationhigh-dimensionalregressionasymptoticoptimalityminimaxlowerbounddebiasedinferencesimultaneousconfidenceintervalsfastgreedyoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes a cross-validation-based method for averaging many high-dimensional penalized estimators (Lasso, SCAD, MCP, and other regularizers) into a single predictor. It claims that when every candidate model is misspecified, the data-driven weights are asymptotically optimal: the averaged estimator's out-of-sample prediction risk converges to the lowest risk achievable by any weight combination, and a new minimax lower bound shows the weight error rate is essentially tight. When at least one candidate model is correct, the method asymptotically places all weight on the correct models, attains a nearly-minimax estimation rate of $O_p(\sqrt{s\log p / n})$ in $\ell_2$ error, and supports simultaneous confidence intervals through a debiased estimator with multiplier bootstrap. The framework allows the number of candidate models $K$ to exceed the sample size $n$ and the dimension $p$ to be much larger than $n$.

What carries the argument

The engine is the simplex-constrained optimization of the $J$-fold cross-validation criterion $CV(w)$ over weights in the probability simplex. The weight-error bounds are carried by a compatibility factor $\phi^2(S^*, B^*, w^*)$, a restricted-eigenvalue-type quantity measuring how well the population candidate matrix $B^*$ separates directions in the cone $\mathcal{C}(S^*, w^*)$, combined with the optimality condition $\langle \mathbb{E}[\frac{\partial L}{\partial\mu} B^{*\top}X], w - w^* \rangle \ge 0$, which removes the constant gradient term that had inflated earlier bounds. The estimation-rate transfer is abstracted by Assumption 3, which packages each candidate estimator's $\ell_2$ and $\ell_1$ errors into the rate $T_{n,q}$; all theorems inherit these rates. The inference block relies on a one-step debiased estimator using a CLIME-type estimate of the inverse Hessian plus Gaussian comparison and anti-concentration bounds. The fast greedy model averaging (FGMA) algorithm majorizes $CV$ by an isotropic quadratic and uses accelerated proximal-gradient steps with backtracking, achieving $CV(\hat w^{(N)})/n = O(N^{-2})$.

What would settle it

Simulate a logistic model with $p > n$ in the all-misspecified regime, compute the cross-validated weights, and track the excess-risk ratio of Theorem 3 against $1 + O_p(\xi_n^{-1} T_{n,q})$. A repeated excess-risk ratio that fails to approach 1 when $\xi_n^{-1} T_{n,q} \to 0$, or a repeated violation of the minimax lower bound (5) at the predicted constants, would refute the core claim. Separately, exhibit a nonconvex SCAD or MCP candidate initialized at a local minimizer whose estimation error exceeds $T_{n,q}(\delta)$ with non-negligible probability, which would break Assumption 3 and the theorems built on it.

Watch

Extended reading notes

Core claim

The paper establishes that $J$-fold cross-validation weight selection, applied to candidate penalized estimators fit with a flexible loss and a general decomposable regularizer, is a near-oracle aggregation rule in high dimensions. Theorem 3 states that when all candidates are misspecified, $(R(\hat w) - R_{\mathrm{Bayes}}) / (\inf_{w} R(w) - R_{\mathrm{Bayes}}) = 1 + O_p(\xi_n^{-1} T_{n,q})$, so the cross-validated average achieves the lowest possible prediction risk asymptotically, with $T_{n,q}$ the uniform estimation error of the candidate fits and $\xi_n$ the gap between the best model-average risk and the Bayes risk. Theorem 2 supplies the first minimax lower bound for the $\ell_1$ weight error, matching the upper bound up to the square root of a compatibility factor. When a correct model is in the candidate set, Theorem 4 shows the sum of weights on correct models tends to one and Corollary 1 delivers $\|\hat\beta_{\mathrm{MA}}(\hat w) - \beta^*\|_2 = O_p(\sqrt{s\log p/n})$, the nearly-minimax Lasso rate. Theorems 6 through 8 provide a Bahadur representation, Gaussian approximation, and multiplier-bootstrap approximation for a one-step debiased post-averaging estimator, enabling simultaneous confidence intervals.

Load-bearing premise

The whole theory depends on the premise that every candidate estimator lies within the declared $\ell_2$ and $\ell_1$ distances of its ideal target with high probability, and this is verified only under extra sparsity and feasibility conditions that can fail for nonconvex estimators and small samples.

Editorial extensions

If this is right

  • When all candidate models are wrong, the cross-validated average is asymptotically as good as the best weight combination, even when the number of candidates and the dimension both exceed the sample size.
  • The $\ell_1$ error of the weight estimator is minimax-optimal up to a square-root compatibility factor, which is the first lower bound of this kind in the model averaging literature.
  • When a correct model is included, all weight concentrates on the correct models and the averaged estimator attains the nearly-minimax $\sqrt{s\log p/n}$ convergence rate in $\ell_2$ error.
  • Post-averaging simultaneous confidence intervals can be constructed via multiplier bootstrap, an inference capability previously unavailable for high-dimensional model averaging.
  • The FGMA algorithm matches the optimal $O(N^{-2})$ rate of first-order convex optimization methods and empirically decreases monotonically, outperforming the prior greedy model averaging algorithms.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The nested-plus-nonnested candidate construction suggests that importance-ranked, partially overlapping candidate sets may beat pure screening-then-grouping; a testable extension is to compare prediction risk against oracle-nested and oracle-disjoint candidate sets in dense-signal regimes.
  • The minimax weight lower bound is proved for quadratic loss and Gaussian design; the same Fano-packing argument might extend to Lipschitz losses by replacing the KL-divergence control with a transport-type bound, though the paper does not make this claim.
  • Because Assumption 3 is load-bearing, practitioners could check candidate-model error rates empirically before trusting the asymptotic-optimality gap; nonconvex fits such as SCAD and MCP may violate the declared rates at moderate sample sizes more often than Lasso fits.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a high-dimensional frequentist model averaging method based on J-fold cross-validation. Candidate models are fit by penalized empirical risk minimization with a general loss and regularizer, and weights are chosen by minimizing a convex CV criterion over the simplex. The theoretical part covers two regimes: when all candidate models are misspecified (non-asymptotic ℓ1 bound for the weight estimator, a minimax lower bound, and asymptotic optimality of the resulting prediction risk), and when a correct model is included (weight consistency, near-oracle estimation rate, and post-averaging debiased inference via Gaussian and multiplier-bootstrap approximation). A fast greedy model averaging algorithm is proposed and analyzed. The claims are supported by detailed proofs in the appendix, simulation studies, and a riboflavin data application.

Significance. If the main results hold as stated, this is a substantial contribution: it extends frequentist model averaging theory to settings where p and K can exceed n, allows general loss functions and regularizers, gives the first minimax lower bound for weight estimation in this literature, and provides post-averaging simultaneous inference for high-dimensional models. The proofs are detailed and the paper ships reproducible code, which are clear strengths. The central caveat is that all of the main theorems are conditional on Assumption 3, and the verification of that assumption for the nonconvex penalties used in the simulations has a gap that is load-bearing for the claimed scope of the method.

major comments (3)
  1. [Assumption 3, Proposition 1, and Section 4.2.1] Assumption 3 is the single most load-bearing ingredient of the paper: Theorems 1, 3, 4, 5, 6, 7 and 8 all inherit the rates T_{n,q}^{(1)} and T_{n,q}^{(2)} from it. Its verification in Proposition 1, however, does not cover the estimator actually computed in the simulations. The proof of Proposition 1 in Appendix C.1 uses the inequality F(Δ̂) ≤ 0, which holds only when β̂_(k) is a global minimizer of the penalized objective. For the folded-concave penalties SCAD and MCP, which satisfy Assumption 4 but are nonconvex, the implemented procedure in Section 4.2.1 uses the LLA algorithm of Fan et al. (2014) and at best produces stationary points or local minima. Nothing in the paper ensures that these solutions coincide with the global argmin, and the ℓ1-feasibility condition ∥β*_(k)∥₁ ≤ R/2 with λ_{n,k}R ≤ c, explicitly required in Proposition 1 for folded-concave penalties, is not guaranteed for the LLA output. Consequently, the theoretical guarantees stated for HDMA with SCAD and MCP penalties in Tables 1–4 and Section 6 do not follow from the current proofs.
  2. [Proposition 1 / Appendix C.1, displayed inequality (C.5)] The ℓ1 error bound for general regularizers under Assumption 4 relies on Lemma 2 in Appendix F.2, which in turn invokes Lemma 4 of Loh and Wainwright (2015) and the ℓ1-ball feasibility condition ∥β̂_(k)∥₁ ≤ R. This condition is stated in Proposition 1 only as an assumption, and no construction is given to guarantee it for the data-dependent choice of λ_{n,k} used in Algorithm 2, namely λ_{n,k} = sqrt(log|A_k|/log p_0) λ_n. Since the tuning parameters are selected by cross-validation in the simulations, it is not clear when the required R and the inequality λ_{n,k}R ≤ c hold simultaneously. The paper should either supply a data-dependent verification of this feasibility condition or state the resulting restrictions explicitly before applying Proposition 1 to the implemented method.
  3. [Theorem 3 and Lemma 7] Theorem 3 states asymptotic optimality in the form (R(ŵ) − R_Bayes)/(inf_w R(w) − R_Bayes) = 1 + O_p(ξ_n^{-1} T_{n,q}), which is exactly the kind of result claimed in the abstract and introduction. The proof via Lemma 7 is sound provided Assumption 3 holds and ξ_n^{-1} T_{n,q} → 0. However, Assumption 6 requires T_{n,q} → 0 and ξ_n^{-1} T_{n,q} → 0, and if T_{n,q} is taken from Proposition 1 with q = 0, the condition T_{n,q} → 0 forces √(s_u log(pK)/n) → 0. In the misspecified case with a fixed number of candidate models and fixed model sizes, the condition is mild, but when K grows exponentially the rate is only logarithmic. The paper should make explicit that the claimed 'first asymptotic optimality result allowing p,K > n' requires Assumption 3 with the stated rates, and that Proposition 1's verification for SCAD/MCP does not currently supply those rates for the implemented estimators.
minor comments (5)
  1. [Title and running head] The word 'Averaging' is split as 'A veraging' in the title and in several section headings; this should be corrected throughout the manuscript.
  2. [References and notation] The name 'Chernozhukov' is misspelled as 'Chernozhuokov' in several places, including the reference list entry for Chernozhuokov et al. (2022); the standard spelling should be used consistently.
  3. [Table 2, Band design, n=100, p=1000, HDMA(SCAD)] The reported standard deviation 0.665 for a mean prediction error of 0.336 appears anomalously large compared to all neighboring entries and may indicate a typo or an outlier influence; please check this entry.
  4. [Algorithm 1] The notation z(N+1) is defined in the loop body using z(N), but the update line also refers to ŵ(N−1); please verify that the indexing is consistent with the standard FISTA-style acceleration step and define the initial z(1) accordingly.
  5. [Appendix A.3] The discussion of non-unique w* is useful, but the sentence ending 'the weights consistency fails' would benefit from a more precise statement distinguishing failure of consistency of ŵ to a particular w* from convergence to the set of minimizers.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; the core theorems are conditional derivations from explicit assumptions, not re-statements of fitted inputs.

full rationale

The derivation chain is self-contained and conditional. Assumption 3 supplies per-candidate non-asymptotic error bounds; Theorem 1 converts them, via restricted strong convexity, a compatibility factor, and an optimality inequality at the simplex boundary, into an oracle inequality for the weight error. Theorem 3 proves asymptotic optimality by comparing the CV criterion to the population risk through uniform empirical-process and Rademacher-complexity bounds, with the misspecification gap ξ_n appearing only as a normalization; the result is not obtained by fitting T_{n,q} and then renaming it as a prediction. The minimax lower bound in Theorem 2 is an independent external construction, and the debiasing/inference results follow standard Bahadur and high-dimensional Gaussian/bootstrap approximation arguments. The self-citations (e.g., Fang et al. 2023, Jiang et al. 2024) are used only for motivation or as related work, not as load-bearing premises, so they do not constitute circularity. The gap highlighted by the skeptical reader—Proposition 1 needing a global argmin and an ℓ1-ball feasibility condition for SCAD/MCP—is a potential mismatch between the implemented LLA algorithm and the theory's assumptions, i.e., a correctness or coverage concern, not a circular reduction of the paper's claims to their own inputs.

Assumptions & free parameters 3 free parameters · 11 assumptions · 0 invented entities

The theory rests on standard high-dimensional estimation assumptions (sub-gaussianity, restricted strong convexity, sparsity of inverse Hessian) and on the high-level Assumption 3 that candidate estimators achieve certain error rates. No new physical or probabilistic entities are introduced.

free parameters (3)
  • K_ne (number of nested candidate models) = 4 in simulations
    Chosen by user, not fitted to data; affects finite-sample behavior, not asymptotic theory.
  • d2 (size of non-nested groups) = 10 in simulations
    Chosen by user; affects candidate model construction.
  • J (number of CV folds) = 5 in simulations
    Chosen by user; theory treats J as fixed.
assumptions (11)
  • domain assumption Assumption 1: loss is convex in second argument with positive second derivative and Lipschitz first derivative.
    Used in Lemma 1 and Theorem 1; standard for convex losses.
  • domain assumption Assumption 2: covariates are sub-gaussian with covariance minimum eigenvalue bounded below by kappa_l > 0.
    Required for restricted strong convexity and concentration arguments.
  • domain assumption Assumption 3: each candidate penalized estimator satisfies uniform l2 and l1 error bounds with high probability.
    Load-bearing; verified under additional conditions in Propositions 1 and 2.
  • domain assumption Assumption 4: regularizer is symmetric, nondecreasing on positive line, with limited concavity.
    Needed for the verification of Assumption 3 for Lasso, SCAD, and MCP.
  • domain assumption Assumption 5: loss is either squared error or Lipschitz in the second argument.
    Used to control the uniform deviation of the empirical loss from its expectation.
  • domain assumption Assumption 6: xi_n^{-1} T_{n,q} -> 0, where xi_n is the approximation error gap.
    Ensures the approximation error dominates the estimation error, a standard condition for asymptotic optimality.
  • domain assumption Assumption 7: either the loss is strongly convex or the candidate models satisfy sparsity conditions.
    Required for weight consistency when correct models are present.
  • domain assumption Assumption 8: inverse Hessian J^{-1} is row-wise sparse.
    Standard condition for debiased inference in high dimensions.
  • domain assumption Assumption 9: second derivative of the loss is Lipschitz in the second argument.
    Controls higher-order terms in the Bahadur representation.
  • domain assumption Assumption 10: eigenvalue lower bounds for the Hessian J and the gradient covariance Lambda.
    Needed for Gaussian and bootstrap approximation results.
  • domain assumption The correct models, when present, are nested within the nested part of the candidate model set.
    Structural assumption in Section 3.2 to identify the weight consistency result.

how reviews work

0 comments
Cite this review

Pith. "Pith review of High-Dimensional Model Averaging via Cross-Validation." pith.science (2026). https://pith.science/paper/Y2NIQUAF

@misc{pith2026250608451,
  author       = {Pith},
  title        = {Pith review of: High-Dimensional Model Averaging via Cross-Validation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/Y2NIQUAF}},
  note         = {Machine review of arXiv:2506.08451}
}
read the original abstract

Model averaging is an important alternative to model selection with attractive prediction accuracy. However, its application to high-dimensional data remains under-explored. We propose a high-dimensional model averaging method via cross-validation under a general framework and systematically establish its theoretical properties. Each candidate model is fitted using a flexible loss function paired with a general regularizer, and the optimal weights are determined by minimizing a cross-validation criterion. When all candidate models are misspecified, we establish a non-asymptotic upper bound and a minimax lower bound for our weight estimator. The asymptotic optimality is also derived, showing that the proposed weight estimator achieves the lowest possible prediction risk asymptotically. When the correct models are included in the candidate model set, the proposed method asymptotically assigns all weights to the correct models, and the model averaging estimator achieves a nearly-oracle convergence rate. Further, we introduce a post-averaging debiased estimator and establish Gaussian and bootstrap approximation to construct simultaneous confidence intervals. A fast greedy model averaging (FGMA) algorithm is proposed to solve the simplex-constrained optimization problem, which has a descent property empirically and a faster convergence rate, compared to the original greedy model averaging algorithm. Empirical results demonstrate the strong competitiveness of the proposed method in prediction and inference, compared to other existing model averaging and selection methods.

Figures

Figures reproduced from arXiv: 2506.08451 by the authors.

Figure 1
Figure 1. We can see that our proposed FGMA algorithm has a descent property and has a [PITH_FULL_IMAGE:figures/full_fig_p026_1.png] view at source ↗
Figure 1
Figure 1. Cross-validation criterion (divided by n) in the first 40 iterations of the proposed FGMA algorithm and the original GMA algorithm for riboflavin data [PITH_FULL_IMAGE:figures/full_fig_p029_1.png] view at source ↗
Figure 2
Figure 2. Out-of-sample prediction errors of different methods for riboflavin data (Bold [PITH_FULL_IMAGE:figures/full_fig_p029_2.png] view at source ↗
Figures from the paper (1 more)
Figure 3
Figure 3. Figure 3: 95% simultaneous confidence intervals of difference methods for riboflavin data [PITH_FULL_IMAGE:figures/full_fig_p030_3.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

73 extracted references · 62 canonical work pages

  1. [1]

    A model-averaging approach for high-dimensional regression

    Tomohiro Ando and Ker-Chau Li. A model-averaging approach for high-dimensional regression. Journal of the American Statistical Association, 109 0 (505): 0 254--265, 2014

    work page 2014

  2. [2]

    A weight-relaxed model averaging approach for high-dimensional generalized linear models

    Tomohiro Ando and Ker-Chau Li. A weight-relaxed model averaging approach for high-dimensional generalized linear models. The Annals of Statistics, 45 0 (6): 0 2654--2679, 2017

    work page 2017

  3. [3]

    Cross-validation: W hat does it estimate and how well does it do it? Journal of the American Statistical Association, 119 0 (546): 0 1434--1445, 2024

    Stephen Bates, Trevor Hastie, and Robert Tibshirani. Cross-validation: W hat does it estimate and how well does it do it? Journal of the American Statistical Association, 119 0 (546): 0 1434--1445, 2024

    work page 2024

  4. [4]

    Distributed testing and estimation under sparse high dimensional models

    Heather Battey, Jianqing Fan, Han Liu, Junwei Lu, and Ziwei Zhu. Distributed testing and estimation under sparse high dimensional models. The Annals of Statistics, 46 0 (3): 0 1352--1382, 2018

    work page 2018

  5. [5]

    A fast iterative shrinkage-thresholding algorithm for linear inverse problems

    Amir Beck and Marc Teboulle. A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM Journal on Imaging Sciences, 2 0 (1): 0 183--202, 2009

    2009

  6. [6]

    Least squares after model selection in high-dimensional sparse models

    Alexandre Belloni and Victor Chernozhukov. Least squares after model selection in high-dimensional sparse models. Bernoulli, 19 0 (2): 0 521--547, 2013

    work page 2013

  7. [7]

    Distributed optimization and statistical learning via the alternating direction method of multipliers

    Stephen Boyd, Neal Parikh, Eric Chu, Borja Peleato, and Jonathan Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning , 3 0 (1): 0 1--122, 2011

    work page 2011

  8. [8]

    High-dimensional inference in misspecified linear models

    Peter B \"u hlmann and Sara Van de Geer. High-dimensional inference in misspecified linear models. Electronic Journal of Statistics, 9 0 (1): 0 1449--1473, 2015

    work page 2015

Show all 73 references
  1. [9]

    High-dimensional statistics with a view toward applications in biology

    Peter B \"u hlmann, Markus Kalisch, and Lukas Meier. High-dimensional statistics with a view toward applications in biology. Annual Review of Statistics and Its Application, 1 0 (1): 0 255--278, 2014

    2014

  2. [10]

    Statistical inference for high-dimensional convoluted rank regression

    Leheng Cai, Xu Guo, Heng Lian, and Liping Zhu. Statistical inference for high-dimensional convoluted rank regression. Journal of the American Statistical Association, 2025. In press

    2025

  3. [11]

    A constrained _1 minimization approach to sparse precision matrix estimation

    Tony Cai, Weidong Liu, and Xi Luo. A constrained _1 minimization approach to sparse precision matrix estimation. Journal of the American Statistical Association, 106 0 (494): 0 594--607, 2011

    2011

  4. [12]

    The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression

    Emmanuel J Cand \`e s and Pragya Sur. The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression. The Annals of Statistics, 48 0 (1): 0 27--42, 2020

    2020

  5. [13]

    Comparison and anti-concentration bounds for maxima of G aussian random vectors

    Victor Chernozhukov, Denis Chetverikov, and Kengo Kato. Comparison and anti-concentration bounds for maxima of G aussian random vectors. Probability Theory and Related Fields, 162: 0 47--70, 2015

    2015

  6. [14]

    Improved central limit theorem and bootstrap approximations in high dimensions

    Victor Chernozhuokov, Denis Chetverikov, Kengo Kato, and Yuta Koike. Improved central limit theorem and bootstrap approximations in high dimensions. The Annals of Statistics, 50 0 (5): 0 2562--2586, 2022

    2022

  7. [15]

    Deviation optimal learning using greedy Q -aggregation

    Dong Dai, Philippe Rigollet, and Tong Zhang. Deviation optimal learning using greedy Q -aggregation. The Annals of Statistics, 40 0 (3): 0 1878--1905, 2012

    1905

  8. [16]

    Assessment and propagation of model uncertainty

    David Draper. Assessment and propagation of model uncertainty. Journal of the Royal Statistical Society Series B: Statistical Methodology, 57 0 (1): 0 45--70, 1995

    1995

  9. [17]

    Efficient projections onto the _1 -ball for learning in high dimensions

    John Duchi, Shai Shalev-Shwartz, Yoram Singer, and Tushar Chandra. Efficient projections onto the _1 -ball for learning in high dimensions. In Proceedings of the 25th International Conference on Machine Learning, pages 272--279, 2008

    2008

  10. [18]

    Variable selection via nonconcave penalized likelihood and its oracle properties

    Jianqing Fan and Runze Li. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96 0 (456): 0 1348--1360, 2001

    2001

  11. [19]

    Strong oracle optimality of folded concave penalized estimation

    Jianqing Fan, Lingzhou Xue, and Hui Zou. Strong oracle optimality of folded concave penalized estimation. The Annals of Statistics, 42 0 (3): 0 819--949, 2014

    2014

  12. [20]

    I- LAMM for sparse learning: Simultaneous control of algorithmic complexity and statistical error

    Jianqing Fan, Han Liu, Qiang Sun, and Tong Zhang. I- LAMM for sparse learning: Simultaneous control of algorithmic complexity and statistical error. The Annals of Statistics, 46 0 (2): 0 814--841, 2018

    2018

  13. [21]

    Communication-efficient accurate statistical estimation

    Jianqing Fan, Yongyi Guo, and Kaizheng Wang. Communication-efficient accurate statistical estimation. Journal of the American Statistical Association, 118 0 (542): 0 1000--1010, 2023

    2023

  14. [22]

    An asymptotic theory for least squares model averaging with nested models

    Fang Fang, Chaoxia Yuan, and Wenling Tian. An asymptotic theory for least squares model averaging with nested models. Econometric Theory, 39 0 (2): 0 412--441, 2023

    2023

  15. [23]

    Least squares model averaging

    Bruce E Hansen. Least squares model averaging. Econometrica, 75 0 (4): 0 1175--1189, 2007

    2007

  16. [24]

    Jackknife model averaging

    Bruce E Hansen and Jeffrey S Racine. Jackknife model averaging. Journal of Econometrics, 167 0 (1): 0 38--46, 2012

    2012

  17. [25]

    Functional martingale residual process for high-dimensional C ox regression with model averaging

    Baihua He, Yanyan Liu, Yuanshan Wu, Guosheng Yin, and Xingqiu Zhao. Functional martingale residual process for high-dimensional C ox regression with model averaging. Journal of Machine Learning Research, 21 0 (207): 0 1--37, 2020

    2020

  18. [26]

    Rank-based greedy model averaging for high-dimensional survival data

    Baihua He, Shuangge Ma, Xinyu Zhang, and Li-Xing Zhu. Rank-based greedy model averaging for high-dimensional survival data. Journal of the American Statistical Association, 118 0 (544): 0 2658--2670, 2023

    2023

  19. [27]

    Bayesian model averaging: A tutorial

    Jennifer A Hoeting, David Madigan, Adrian E Raftery, and Chris T Volinsky. Bayesian model averaging: A tutorial. Statistical Science, 14 0 (4): 0 382--417, 1999

    1999

  20. [28]

    A tutorial on MM algorithms

    David R Hunter and Kenneth Lange. A tutorial on MM algorithms. The American Statistician, 58 0 (1): 0 30--37, 2004

    2004

  21. [29]

    Confidence intervals and hypothesis testing for high-dimensional regression

    Adel Javanmard and Andrea Montanari. Confidence intervals and hypothesis testing for high-dimensional regression. Journal of Machine Learning Research, 15 0 (1): 0 2869--2909, 2014

    2014

  22. [30]

    Robust model averaging prediction of longitudinal response with ultrahigh-dimensional covariates

    Binyan Jiang, Jing Lv, Jialiang Li, and Ming-Yen Cheng. Robust model averaging prediction of longitudinal response with ultrahigh-dimensional covariates. Journal of the Royal Statistical Society Series B: Statistical Methodology, 87 0 (2): 0 337--361, 2024

    2024

  23. [31]

    Probability in Banach Spaces: Isoperimetry and Processes, volume 23

    Michel Ledoux and Michel Talagrand. Probability in Banach Spaces: Isoperimetry and Processes, volume 23. Springer Science & Business Media, 1991

    1991

  24. [32]

    Information theory and mixing least-squares regressions

    Gilbert Leung and Andrew R Barron. Information theory and mixing least-squares regressions. IEEE Transactions on Information Theory, 52 0 (8): 0 3396--3410, 2006

    2006

  25. [33]

    Estimation and inference for high-dimensional generalized linear models with knowledge transfer

    Sai Li, Linjun Zhang, T Tony Cai, and Hongzhe Li. Estimation and inference for high-dimensional generalized linear models with knowledge transfer. Journal of the American Statistical Association, 119 0 (546): 0 1274--1285, 2024

    2024

  26. [34]

    Distribution theory of the least squares averaging estimator

    Chu-An Liu. Distribution theory of the least squares averaging estimator. Journal of Econometrics, 186 0 (1): 0 142--159, 2015

    2015

  27. [35]

    Communication-efficient distributed sparse learning with oracle property and geometric convergence

    Weidong Liu, Xiaojun Mao, and Jiyuan Tu. Communication-efficient distributed sparse learning with oracle property and geometric convergence. Journal of the American Statistical Association, 2025. In press

    2025

  28. [36]

    Regularized M -estimators with nonconvexity: Statistical and algorithmic theory for local optima

    Po-Ling Loh and Martin J Wainwright. Regularized M -estimators with nonconvexity: Statistical and algorithmic theory for local optima. Journal of Machine Learning Research, 16 0 (1): 0 559--616, 2015

    2015

  29. [37]

    Model selection principles in misspecified models

    Jinchi Lv and Jun S Liu. Model selection principles in misspecified models. Journal of the Royal Statistical Society Series B: Statistical Methodology, 76 0 (1): 0 141--167, 2014

    2014

  30. [38]

    P-values for high-dimensional regression

    Nicolai Meinshausen, Lukas Meier, and Peter B \"u hlmann. P-values for high-dimensional regression. Journal of the American Statistical Association, 104 0 (488): 0 1671--1681, 2009

    2009

  31. [39]

    A unified framework for high-dimensional analysis of M -estimators with decomposable regularizers

    Sahand N Negahban, Pradeep Ravikumar, Martin J Wainwright, and Bin Yu. A unified framework for high-dimensional analysis of M -estimators with decomposable regularizers. Statistical Science, 27 0 (4): 0 538--557, 2012

    2012

  32. [40]

    Gradient methods for minimizing composite functions

    Yu Nesterov. Gradient methods for minimizing composite functions. Mathematical Programming, 140 0 (1): 0 125--161, 2013

    2013

  33. [41]

    Lectures on Convex Optimization, volume 137

    Yurii Nesterov. Lectures on Convex Optimization, volume 137. Springer, 2018

    2018

  34. [42]

    A general theory of hypothesis tests and confidence regions for sparse high dimensional models

    Yang Ning and Han Liu. A general theory of hypothesis tests and confidence regions for sparse high dimensional models. The Annals of Statistics, 45 0 (1): 0 158--195, 2017

    2017

  35. [43]

    Proximal algorithms

    Neal Parikh and Stephen Boyd. Proximal algorithms. Foundations and Trends in Optimization , 1 0 (3): 0 127--239, 2014

    2014

  36. [44]

    On improvability of model selection by model averaging

    Jingfu Peng and Yuhong Yang. On improvability of model selection by model averaging. Journal of Econometrics, 229 0 (2): 0 246--262, 2022

    2022

  37. [45]

    On optimality of M allows model averaging

    Jingfu Peng, Yang Li, and Yuhong Yang. On optimality of M allows model averaging. Journal of the American Statistical Association, 2024

    2024

  38. [46]

    Racine, Qi Li, Dalei Yu, and Li Zheng and

    Jeffrey S. Racine, Qi Li, Dalei Yu, and Li Zheng and. Optimal model averaging of mixed-data kernel-weighted spline regressions. Journal of Business & Economic Statistics, 41 0 (4): 0 1251--1261, 2023

    2023

  39. [47]

    A scalable estimate of the out-of-sample prediction error via approximate leave-one-out cross-validation

    Kamiar Rahnama Rad and Arian Maleki. A scalable estimate of the out-of-sample prediction error via approximate leave-one-out cross-validation. Journal of the Royal Statistical Society Series B: Statistical Methodology, 82 0 (4): 0 965--996, 2020

    2020

  40. [48]

    Minimax rates of estimation for high-dimensional linear regression over l_q -balls

    Garvesh Raskutti, Martin J Wainwright, and Bin Yu. Minimax rates of estimation for high-dimensional linear regression over l_q -balls. IEEE Transactions on Information Theory, 57 0 (10): 0 6976--6994, 2011

    2011

  41. [49]

    Regression shrinkage and selection via the L asso

    Robert Tibshirani. Regression shrinkage and selection via the L asso. Journal of the Royal Statistical Society Series B: Statistical Methodology, 58 0 (1): 0 267--288, 1996

    1996

  42. [50]

    Quantile prediction with factor-augmented regression: Structural instability and model uncertainty

    Yundong Tu and Siwei Wang. Quantile prediction with factor-augmented regression: Structural instability and model uncertainty. Journal of Econometrics, 249: 0 105999, 2025

    2025

  43. [51]

    On asymptotically optimal confidence regions and tests for high-dimensional models

    Sara Van de Geer, Peter B \"u hlmann, Ya’acov Ritov, and Ruben Dezeure. On asymptotically optimal confidence regions and tests for high-dimensional models. The Annals of Statistics, 42 0 (3): 0 1166--1202, 2014

    2014

  44. [52]

    On the conditions used to prove oracle results for the L asso

    Sara A Van de Geer and Peter B \"u hlmann. On the conditions used to prove oracle results for the L asso. Electronic Journal of Statistics, 3: 0 1360--1392, 2009

    2009

  45. [53]

    High-Dimensional Probability: An Introduction with Applications in Data Science, volume 47

    Roman Vershynin. High-Dimensional Probability: An Introduction with Applications in Data Science, volume 47. Cambridge University Press, 2018

    2018

  46. [54]

    High-Dimensional Statistics: A Non-Asymptotic Viewpoint, volume 48

    Martin J Wainwright. High-Dimensional Statistics: A Non-Asymptotic Viewpoint, volume 48. Cambridge University Press, 2019

    2019

  47. [55]

    Jackknife model averaging for high-dimensional quantile regression

    Miaomiao Wang, Xinyu Zhang, Alan TK Wan, Kang You, and Guohua Zou. Jackknife model averaging for high-dimensional quantile regression. Biometrics, 79 0 (1): 0 178--189, 2023

    2023

  48. [56]

    Maximum likelihood estimation of misspecified models

    Halbert White. Maximum likelihood estimation of misspecified models. Econometrica, 50 0 (1): 0 1--25, 1982

    1982

  49. [57]

    Valid post-averaging inference in AR-G/GARCH models

    Hsin-Chieh Wong. Valid post-averaging inference in AR-G/GARCH models. Journal of Time Series Analysis, 2025. In press

    2025

  50. [58]

    A model-averaging method for high-dimensional regression with missing responses at random

    Jinhan Xie, Xiaodong Yan, and Niansheng Tang. A model-averaging method for high-dimensional regression with missing responses at random. Statistica Sinica, 31 0 (2): 0 1005--1026, 2021

    2021

  51. [59]

    Optimal model averaging forecasting in high-dimensional survival analysis

    Xiaodong Yan, Hongni Wang, Wei Wang, Jinhan Xie, Yanyan Ren, and Xinjun Wang. Optimal model averaging forecasting in high-dimensional survival analysis. International Journal of Forecasting, 37 0 (3): 0 1147--1155, 2021

    2021

  52. [60]

    Confidence intervals and hypothesis testing for high-dimensional quantile regression: Convolution smoothing and debiasing

    Yibo Yan, Xiaozhou Wang, and Riquan Zhang. Confidence intervals and hypothesis testing for high-dimensional quantile regression: Convolution smoothing and debiasing. Journal of Machine Learning Research, 24 0 (245): 0 1--49, 2023

    2023

  53. [61]

    Adaptive regression by mixing

    Yuhong Yang. Adaptive regression by mixing. Journal of the American Statistical Association, 96 0 (454): 0 574--588, 2001

    2001

  54. [62]

    Post-averaging inference for optimal model averaging estimator in generalized linear models

    Dalei Yu, Heng Lian, Yuying Sun, Xinyu Zhang, and Yongmiao Hong. Post-averaging inference for optimal model averaging estimator in generalized linear models. Econometric Reviews, 43 0 (2-4): 0 98--122, 2024

    2024

  55. [63]

    Combining linear regression models: When and how? Journal of the American Statistical Association, 100 0 (472): 0 1202--1214, 2005

    Zheng Yuan and Yuhong Yang. Combining linear regression models: When and how? Journal of the American Statistical Association, 100 0 (472): 0 1202--1214, 2005

    2005

  56. [64]

    Nearly unbiased variable selection under minimax concave penalty

    Cun-Hui Zhang. Nearly unbiased variable selection under minimax concave penalty. The Annals of Statistics, 38 0 (2): 0 894--942, 2010

    2010

  57. [65]

    Confidence intervals for low dimensional parameters in high dimensional linear models

    Cun-Hui Zhang and Stephanie S Zhang. Confidence intervals for low dimensional parameters in high dimensional linear models. Journal of the Royal Statistical Society Series B: Statistical Methodology, 76 0 (1): 0 217--242, 2014

    2014

  58. [66]

    A general theory of concave regularization for high-dimensional sparse estimation problems

    Cun-Hui Zhang and Tong Zhang. A general theory of concave regularization for high-dimensional sparse estimation problems. Statistical Science, 27 0 (4): 0 576--593, 2012

    2012

  59. [67]

    Concentration inequalities for statistical inference

    Huiming Zhang and Song Xi Chen. Concentration inequalities for statistical inference. arXiv preprint, arXiv:2011.02258, 2020

    2011 arXiv

  60. [68]

    Inference after model averaging in linear regression models

    Xinyu Zhang and Chu-An Liu. Inference after model averaging in linear regression models. Econometric Theory, 35 0 (4): 0 816--841, 2019

    2019

  61. [69]

    Model averaging prediction by K -fold cross-validation

    Xinyu Zhang and Chu-An Liu. Model averaging prediction by K -fold cross-validation. Journal of Econometrics, 235 0 (1): 0 280--301, 2023

    2023

  62. [70]

    Optimal model averaging estimation for generalized linear models and generalized linear mixed-effects models

    Xinyu Zhang, Dalei Yu, Guohua Zou, and Hua Liang. Optimal model averaging estimation for generalized linear models and generalized linear mixed-effects models. Journal of the American Statistical Association, 111 0 (516): 0 1775--1790, 2016

    2016

  63. [71]

    Parsimonious model averaging with a diverging number of parameters

    Xinyu Zhang, Guohua Zou, Hua Liang, and Raymond J Carroll. Parsimonious model averaging with a diverging number of parameters. Journal of the American Statistical Association, 115 0 (530): 0 972--984, 2020

    2020

  64. [72]

    Regularization and variable selection via the elastic net

    Hui Zou and Trevor Hastie. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society Series B: Statistical Methodology, 67 0 (2): 0 301--320, 2005

    2005

  65. [73]

    One-step sparse estimates in nonconcave penalized likelihood models

    Hui Zou and Runze Li. One-step sparse estimates in nonconcave penalized likelihood models. The Annals of Statistics, 36 0 (4): 0 1509--1533, 2008

    2008

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.