Pith. sign in

REVIEW 2 major objections 2 minor 1 cited by

Combining pre-trained models via localized model averaging

T0 review · 2 major / 2 minor · reviewed 2026-05-14 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Modeling averaging weights as functions of covariates yields asymptotically optimal in-sample and out-of-sample risks when combining pre-trained models.

arxiv 2605.13421 v1 pith:SZFWBOAT submitted 2026-05-13 stat.ME

Ziwen Gao , Baihua He , Yuhong Yang This is my paper
classification stat.ME
keywords localizedmodelaveragingpre-trainedmodelsasymptoticoptimalitycovariate-dependentweightsweightconsistencygenerallossframeworkcombinationrisk
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a localized model averaging method in which the weights assigned to different pre-trained models are learned as flexible functions of the input covariates. This formulation lets the averaging procedure adapt to the fact that different models perform better in different contexts. The authors work under a general loss that covers many prediction tasks and prove that the resulting risks are asymptotically optimal both inside and outside the training sample while the estimated weights remain consistent. A sympathetic reader cares because fixed-weight averaging cannot capture how relative model strengths shift with the data, and the localized approach directly addresses that limitation.

What carries the argument

Localized weights expressed as functions of covariates and learned under a general loss.

What would settle it

A dataset or simulation in which the estimated weights fail to converge to the optimal local weights or the achieved risk stays a fixed amount above the oracle risk as sample size grows.

Watch

Extended reading notes

Core claim

We introduce localized model averaging where the weights are modeled as functions of the covariates, allowing the procedure to capture varying relative advantages of pre-trained models across heterogeneous contexts. Under a general loss framework, we establish asymptotic optimality for both in-sample and out-of-sample risks together with consistency of the estimated weights.

Load-bearing premise

The data conditions permit consistent estimation of the covariate-dependent local weights under the chosen general loss.

Editorial extensions

If this is right

  • The averaging procedure adapts automatically to changes in input context.
  • Both in-sample and out-of-sample risks converge to the best attainable level.
  • The estimated weights are consistent for the true optimal local weights.
  • The same framework applies across a wide range of prediction tasks via the general loss.
  • No fixed set of weights is required when model rankings shift with covariates.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The same localized-weight idea could be tested on ensembles of fine-tuned models rather than only off-the-shelf pre-trained ones.
  • Implementation would require only that the weight functions be parameterized flexibly enough to capture the relevant covariate effects.
  • If the consistency result holds, practitioners could replace manual model selection with a single fitted weight surface.
  • Extensions to streaming or non-stationary data would need to check whether the same asymptotic arguments still apply.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 2 minor

Summary. The manuscript proposes a localized model averaging procedure for combining pre-trained models, in which the averaging weights are modeled as flexible functions of the covariates rather than global constants. Under a general loss framework, the authors claim to establish asymptotic optimality of the resulting estimator for both in-sample and out-of-sample risks together with consistency of the estimated local weights, and they support these claims with numerical experiments on synthetic and real data.

Significance. If the asymptotic results are rigorously established, the work would provide a statistically grounded method for adaptive combination of pre-trained models that respects heterogeneity in covariate space, extending classical model averaging to settings where relative model performance varies locally. The general-loss formulation and out-of-sample optimality claim would be particularly useful for modern prediction pipelines.

major comments (2)
  1. [§3.2, Theorem 3.2] §3.2, Theorem 3.2: the out-of-sample asymptotic optimality result requires uniform convergence of the nonparametric local-weight estimators over the entire covariate support, yet the stated regularity conditions do not explicitly include the Hölder smoothness order of the weight functions or the precise bandwidth rates needed to guarantee the uniform rate; without these, the oracle-risk property may fail in regions of low design density.
  2. [Assumption 2.3] Assumption 2.3 and the proof of consistency: the conditions allowing consistent estimation of the local weights under a general loss are given, but it is not shown that these conditions are sufficient to control the remainder term when the loss is non-smooth or when the covariate density is unbounded, which is load-bearing for the claimed out-of-sample optimality.
minor comments (2)
  1. [§2] The notation for the local weight functions w_k(x) is introduced without an explicit statement of the dimension of x or the support of the covariate distribution, which affects readability of the subsequent convergence arguments.
  2. [§5] In the numerical experiments, the tables reporting risk values do not include standard errors or the number of Monte Carlo replications, making it difficult to assess the statistical significance of the reported improvements.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive comments, which help strengthen the rigor of our asymptotic results. We address each major comment below and will revise the manuscript to incorporate the necessary clarifications and additions.

read point-by-point responses

  1. Referee: [§3.2, Theorem 3.2] §3.2, Theorem 3.2: the out-of-sample asymptotic optimality result requires uniform convergence of the nonparametric local-weight estimators over the entire covariate support, yet the stated regularity conditions do not explicitly include the Hölder smoothness order of the weight functions or the precise bandwidth rates needed to guarantee the uniform rate; without these, the oracle-risk property may fail in regions of low design density.

    Authors: We agree that the regularity conditions in the manuscript are incomplete for guaranteeing uniform convergence over the full covariate support. In the revised version, we will explicitly augment the assumptions to include the Hölder smoothness order α of the weight functions and specify the bandwidth rates (e.g., h_n = O(n^{-1/(2α + d)}) with n h_n^d → ∞) required for the uniform rate. We will add a supporting lemma establishing sup-norm convergence of the local-weight estimators, incorporating standard trimming or boundary corrections to handle low-density regions, thereby ensuring the oracle-risk property holds uniformly. revision: yes

  2. Referee: [Assumption 2.3] Assumption 2.3 and the proof of consistency: the conditions allowing consistent estimation of the local weights under a general loss are given, but it is not shown that these conditions are sufficient to control the remainder term when the loss is non-smooth or when the covariate density is unbounded, which is load-bearing for the claimed out-of-sample optimality.

    Authors: The referee correctly notes that the current proof does not explicitly bound the remainder term under non-smooth losses or unbounded densities. We will revise the proof of consistency under Assumption 2.3 to include these controls: we will add the assumption that the loss is uniformly Lipschitz continuous (standard for general losses and sufficient to handle non-smoothness) and restrict attention to compact sets where the covariate density is bounded away from zero and infinity, with a brief discussion of tail truncation for unbounded cases. These additions will make the out-of-sample optimality claim rigorous. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity in derivation chain

full rationale

The paper proposes localized model averaging with covariate-dependent weights under a general loss, then claims to establish asymptotic optimality for in-sample/out-of-sample risks plus weight consistency via theoretical analysis. No steps reduce by construction to fitted inputs, self-definitions, or load-bearing self-citations; the optimality follows from standard consistency arguments under stated data conditions rather than renaming or smuggling ansatzes. The derivation remains self-contained against external benchmarks.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

Abstract-only review limits visibility; method rests on standard statistical assumptions for asymptotic results and a general loss framework.

assumptions (2)
  • domain assumption General loss framework accommodates broad class of prediction tasks
    Explicitly stated as allowing flexible local weights under general loss.
  • domain assumption Data distribution permits consistent estimation of local weights
    Required for the claimed consistency of estimated weights.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Combining pre-trained models via localized model averaging." pith.science (2026). https://pith.science/paper/SZFWBOAT

@misc{pith2026260513421,
  author       = {Pith},
  title        = {Pith review of: Combining pre-trained models via localized model averaging},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SZFWBOAT}},
  note         = {Machine review of arXiv:2605.13421}
}
read the original abstract

Many pre-trained models (PTMs) are available in modern applications. Because different PTMs are often trained on different datasets, their performances can vary substantially for different new tasks, and the ranking of the candidates may depend heavily on the input. Motivated by this, we propose a localized model averaging method with weights modeled as functions of the covariates, making it substantially more versatile than existing model averaging methods. This formulation allows the model averaging procedure to adaptively capture the varying relative advantages of different PTMs across heterogeneous contexts. Specifically, we learn flexible local weights under a general loss framework that accommodates a broad class of prediction tasks. We further establish the asymptotic optimality of the proposed method for both in-sample and out-of-sample risks, as well as the consistency of the estimated weights. Extensive numerical experiments further demonstrate the effectiveness of the proposed method.

Figures

Figures reproduced from arXiv: 2605.13421 by the authors.

Figure 1
Figure 1. A motivating example. In machine learning, there is a similar idea known as the mixture of experts (MoE) method. The MoE framework proposed by Jacobs et al. (1991) involves a form of model averaging. MoE consists of a set of experts and a gating network, where the gating network dynamically adjusts the weights according to X to combine the predictions from multiple experts. MoE has been widely applied in large langu… view at source ↗
Figure 2
Figure 2. The true weight functions and the estimated weight functions in setting S1. [PITH_FULL_IMAGE:figures/full_fig_p017_2.png] view at source ↗
Figure 3
Figure 3. Comparison of MSPE for different methods. [PITH_FULL_IMAGE:figures/full_fig_p021_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Comparison of the classification accuracy for different methods. [PITH_FULL_IMAGE:figures/full_fig_p023_4.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Optimal Mixture-of-Experts Model Averaging for Conditional Generative Models

    stat.ML 2026-07 accept novelty 6.5 of 10

    Sample-based MMD model averaging of conditional generators is asymptotically optimal, and input-adaptive MoEMA weights improve over fixed averaging and single models across modalities.

Reference graph

Works this paper leans on

202 extracted references · 202 canonical work pages · cited by 1 Pith paper

  1. [1]

    The Annals of Statistics , volume=

    Functional aggregation for nonparametric regression , author=. The Annals of Statistics , volume=. 2000 , publisher=

    work page 2000

  2. [2]

    Combining forecasting procedures:

    Yang, Yuhong , journal=. Combining forecasting procedures:. 2004 , publisher=

    work page 2004

  3. [3]

    Journal of Econometrics , volume=

    Adaptively combined forecasting for discrete response time series , author=. Journal of Econometrics , volume=. 2013 , publisher=

    work page 2013

  4. [4]

    Journal of the American Statistical Association , volume=

    Adaboost semiparametric model averaging prediction for multiple categories , author=. Journal of the American Statistical Association , volume=. 2022 , publisher=

    work page 2022

  5. [5]

    Economics Letters , year=2017, volume=

    Xie, Tian , title=. Economics Letters , year=2017, volume=

    work page 2017

  6. [6]

    Model averaging based on

    Zhang, Xinyu and Zou, Guohua and Carroll, Raymond J , journal=. Model averaging based on. 2015 , publisher=

    work page 2015

  7. [7]

    Economics Letters , volume=

    Prediction model averaging estimator , author=. Economics Letters , volume=. 2015 , publisher=

    work page 2015

  8. [8]

    Journal of Econometrics , volume=

    Toward optimal model averaging in regression models with time series errors , author=. Journal of Econometrics , volume=. 2015 , publisher=

    work page 2015

Show all 202 references
  1. [9]

    Journal of Applied Econometrics , pages=

    Feasible cross-validatory model selection for general stationary processes , author=. Journal of Applied Econometrics , pages=. 1997 , publisher=

    1997

  2. [10]

    1952 , publisher=

    Inequalities , author=. 1952 , publisher=

    1952

  3. [11]

    , title =

    Liu, C.-A. , title =. Journal of Econometrics , year =

  4. [12]

    Carroll , title =

    Hua Liang, and Suojin Wang, and Raymond J. Carroll , title =. Biometrika , year =

  5. [13]

    and Linton, O

    Li, D. and Linton, O. and Lu, Z. , title =. Journal of Econometrics , year =

  6. [14]

    Journal of Nonparametric Statistics , year =

    Na Li, and Xingzhong Xu, and Pei Jin , title =. Journal of Nonparametric Statistics , year =

  7. [15]

    Hansen, B. E. , title =. Quantitative Economics , year =

  8. [16]

    IEEE Transactions on Information Theory , volume=

    Information theory and mixing least-squares regressions , author=. IEEE Transactions on Information Theory , volume=. 2006 , publisher=

    2006

  9. [17]

    Review of Finance , year =

    Dieckmann, Stephan and Plank, Thomas , title =. Review of Finance , year =

  10. [18]

    Journal of the American Statistical Association , volume=

    Semiparametric estimates of the relation between weather and electricity sales , author=. Journal of the American Statistical Association , volume=. 1986 , publisher=

    1986

  11. [19]

    Journal of Multivariate Analysis , volume=

    Local linear estimation in partly linear models , author=. Journal of Multivariate Analysis , volume=. 1997 , publisher=

    1997

  12. [20]

    and Su, L

    Lu, X. and Su, L. , year =. Jackknife model averaging for quantile regressions , journal =

  13. [21]

    2000 , publisher=

    Partially linear models , author=. 2000 , publisher=

    2000

  14. [22]

    Magnus, J. R. and Wan, A. T. K. and Zhang, X. , journal=. Weighted average least squares estimation with nonspherical disturbances and an application to the. 2011 , publisher=

    2011

  15. [23]

    Theory of Probability & Its Applications , volume=

    Bounds for the moments of linear and quadratic forms in independent variables , author=. Theory of Probability & Its Applications , volume=. 1960 , publisher=

    1960

  16. [24]

    Econometrica , volume =

    Root-N-Consistent Semiparametric Regression , author =. Econometrica , volume =

  17. [25]

    Econometric Theory , year=2005, volume=

    Juhl, Ted and Xiao, Zhijie , title=. Econometric Theory , year=2005, volume=

    2005

  18. [26]

    Journal of Econometrics , year=1996, volume=

    Li, Qi and Stengos, Thanasis , title=. Journal of Econometrics , year=1996, volume=

    1996

  19. [27]

    Journal of Econometrics , year=2010, volume=

    Su, Liangjun and Jin, Sainan , title=. Journal of Econometrics , year=2010, volume=

    2010

  20. [28]

    Econometric Theory , year=2010, volume=

    Su, Liangjun and White, Halbert , title=. Econometric Theory , year=2010, volume=

    2010

  21. [29]

    , title=

    Li, Qi and Wooldridge, Jeffrey M. , title=. Econometric Theory , year=2002, volume=

    2002

  22. [30]

    Annals of Economics and Finance , year=2005, volume=

    Yiguo Sun , title=. Annals of Economics and Finance , year=2005, volume=

    2005

  23. [31]

    Econometric Theory , volume =

    Lee,Sokbae , title =. Econometric Theory , volume =. 2003 , pages =

    2003

  24. [32]

    Computational Statistics & Data Analysis , volume =

    Hua Liang , title =. Computational Statistics & Data Analysis , volume =. 2006 , pages =

    2006

  25. [33]

    Krief , title =

    Jerome M. Krief , title =. Econometric Theory , volume =. 2013 , pages =

    2013

  26. [34]

    Journal of the Royal Statistical Society

    Spline Smoothing in a Partly Linear Model , author =. Journal of the Royal Statistical Society. Series B (Methodological) , volume =

  27. [35]

    and Magnus, Jan R

    Abadir, Karim M. and Magnus, Jan R. , title =

  28. [36]

    Gijbels , title =

    Jianqing Fan, and I. Gijbels , title =

  29. [37]

    Estimation in a semiparametric partially linear errors-in-variables model , author=

  30. [38]

    Econometrica , volume =

    Household Gasoline Demand in the United States , author =. Econometrica , volume =

  31. [39]

    Variable Selection in Nonparametric Varying-Coefficient Models for Analysis of Repeated Measurements , author=

  32. [40]

    Statistica Sinica , volume=

    Moment-Based Method For Random Effects Selection In Linear Mixed Models , author=. Statistica Sinica , volume=

  33. [41]

    New Estimation and Model Selection Procedures for Semiparametric Modeling in Longitudinal Data Analysis , author=

  34. [42]

    Variable selection in semiparametric regression modeling , author=

  35. [43]

    Akaike , title =

    H. Akaike , title =. 1973 , journal =

    1973

  36. [44]

    Operations Research Quarterly , volume =

    The Combination of Forecasts , author =. Operations Research Quarterly , volume =

  37. [45]

    Biometrics , volume=

    Joint Variable Selection for Fixed and Random Effects in Linear Mixed-Effects Models , author=. Biometrics , volume=. 2010 , publisher=

    2010

  38. [46]

    and Carroll, R

    Claeskens, G. and Carroll, R. J. , year = 2007, title =

    2007

  39. [47]

    and Croux, C

    Claeskens, G. and Croux, C. and. Variable selection for logistic regression using a prediction-focused information criterion , journal = bioc, volume = 62, number=4, pages =

  40. [48]

    and Hjort, N

    Claeskens, G. and Hjort, N. L. , title =

  41. [49]

    P. J. Green, and Bernard. W. Silverman , title =

  42. [50]

    Donohue, M. C. and Overholser, R. and Xu, R. and Vaida, F. , title =. Biometrika , volume =. 2011 , pages =

    2011

  43. [51]

    and Kneib, T

    Greven, S. and Kneib, T. , title =. 2010 , pages =

    2010

  44. [52]

    , title=

    Chang, Roberto and Kaltani, Linda and Loayza, Norman V. , title=. Journal of Development Economics , year=2009, volume=

    2009

  45. [53]

    Hjort, N. L. and Claeskens, G. , year = 2003, title =

    2003

  46. [54]

    Biometrics , volume=

    Fixed and random effects selection in mixed effects models , author=. Biometrics , volume=. 2011 , publisher=

    2011

  47. [55]

    A forecast comparison of residential housing prices by parametric versus semiparametric conditional mean estimators , journal=

    Gen. A forecast comparison of residential housing prices by parametric versus semiparametric conditional mean estimators , journal=

  48. [56]

    , journal=

    Andrews, D.W.K. , journal=. Asymptotic optimality of generalized. 1991 , publisher=

    1991

  49. [57]

    Biometrika , volume=

    Single-index model selections , author=. Biometrika , volume=. 2001 , publisher=

    2001

  50. [58]

    and Wu, H

    Liang, H. and Wu, H. and Zou, G. , year = 2008, title =

    2008

  51. [59]

    and Zou, G

    Liang, H. and Zou, G. and Wan, A. T. K. and Zhang, X. , year =. Optimal weight choice for frequentist model average estimators , journal = jasa, volume =

  52. [60]

    Journal of Applied Econometrics , volume =

    Tan, Chih Ming , title =. Journal of Applied Econometrics , volume =

  53. [61]

    Magnus, and Wendun Wang , title =

    Jan R. Magnus, and Wendun Wang , title =. Oxford Bulletin of Economics and Statistics , year =

  54. [62]

    Determinants of Long-Term Growth: A

    Xavier. Determinants of Long-Term Growth: A. American Economic Review , volume =

  55. [63]

    McMahon and M

    D. McMahon and M. Lederman and D. W. Haas and R. Haubrich and J. Stanford and E. Cooney and J. Horton and D. Kelleher and L. Ross and A. Cutrell and D. Lee and W. Spreen and J. W. Mellors , title =. Antiviral Therapy , year =

  56. [64]

    Berger , title =

    James O. Berger , title =

  57. [65]

    Miller, A. J. , title =

  58. [66]

    David Ruppert, and M. P. Wand, and R. J. Carroll , title =

  59. [67]

    N., and Johnson, P

    Durlauf, S. N., and Johnson, P. A., and Temple, J. R. W , title =. Handbook of Economic Growth , address =

  60. [68]

    Adonis Yatchew , title =

  61. [69]

    Yatchew , title=

    A. Yatchew , title=. Journal of Applied Econometrics , year=2000, volume=

    2000

  62. [70]

    Econometrica , year=2001, volume=

    Adonis Yatchew and Joungyeo Angela No , title=. Econometrica , year=2001, volume=

    2001

  63. [71]

    and Niu, X.-F

    Pu, W. and Niu, X.-F. , title =. Journal of Multivariate Analysis , volume =. 2006 , pages =

    2006

  64. [72]

    Robinson, G. K. , title =. Statistical Science , volume =. 1991 , pages =

    1991

  65. [73]

    Schwarz , title =

    G. Schwarz , title =. 1978 , volume =

    1978

  66. [74]

    and Blanchard, S

    Vaida, F. and Blanchard, S. , title =. Biometrika , volume =. 2005 , pages =

    2005

  67. [75]

    Wan, A. T. K. and Zhang, X. and Zou, G. , title =. Journal of Econometrics , year =

  68. [76]

    , title =

    Yang, Y. , title =. 2001 , number =

    2001

  69. [77]

    , title =

    Yang, Y. , title =. Statistica Sinica , volume =. 2003 , pages =

    2003

  70. [78]

    and Yang, Y

    Yuan, Z. and Yang, Y. , title =. 2005 , number=

    2005

  71. [79]

    and Liang, H

    Zhang, X. and Liang, H. , year =. Focused information criterion and model averaging for generalized additive partial linear models , journal = annals, volume =

  72. [80]

    and Wan, A

    Zhang, X. and Wan, A. T. K. and Zhou, S. Z. , title =. Journal of Business & Economic Statistics , volume =. 2012 , pages =

    2012

  73. [81]

    Ho, D. D. and Neumann, A. U. and Perelson, A. S. and Chen, W. and Leonard, J. M. and Markowitz, M. , title =. Nature , volume = 373, pages =

  74. [82]

    and Ghosh, S

    Wei, X. and Ghosh, S. K. and Taylor, M. E. and Johnson, V. A. and Emini, E. A. and Deutsch, P. and Lifson, J. D. and Bonhoeffer, S. and Nowak, M. A. and Hahn, B. H. and Saag, M.S. and and Shaw, G. M. , year = 1995, title =. Nature , volume = 373, pages =

    1995

  75. [83]

    Wu, H. L. and Ding, A. A. , year = 1999, title =

    1999

  76. [84]

    Laird, N. M. and Ware, J. H. , title =

  77. [85]

    Lederman, M. M. and Connick, E. and Landay, A. and Kuritzkes, D. R. and Spritzkes, J. and St Clair, M. and Kotzin, B. L. and Fox, L. and Chiozzi, M. H. and Leonard, J. M. and Rousseau, F. and Wade, M. and Roe, J. D. and Martinez, A. and Kessler, H. , year = 1998, title =. The ...

    1998

  78. [86]

    Journal of the Royal Statistical Society

    Kernel smoothing in partial linear models , author=. Journal of the Royal Statistical Society. Series B (Methodological) , volume = 50, pages =

  79. [87]

    A plug-in averaging estimator for regressions with heteroskedastic errors , author=

  80. [88]

    Carroll , title =

    Li Wang, and Xiang Liu, and Hua Liang, and Raymond J. Carroll , title =. The Annals of Statistics , volume =. 2011 , pages =

    2011

  81. [89]

    Journal of the American Statistical Association , volume =

    Hua Liang, and Runze Li , title =. Journal of the American Statistical Association , volume =. 2009 , pages =

    2009

  82. [90]

    Danilov, Dmitry and Magnus, J. R. , title=. Journal of Econometrics , year=2004, volume=

    2004

  83. [91]

    J. R. Magnus, and Wendun Wang, and Xinyu Zhang , title=. Econometric Reviews , year=

  84. [92]

    Journal of the American Statistical Association , volume =

    Xiao Ni, and Hao Helen Zhang, and Daowen Zhang , title =. Journal of the American Statistical Association , volume =. 2009 , pages =

    2009

  85. [93]

    Journal of Statistical Planning and Inference , volume =

    Ying-Zi Fu, and Xue-Dong Chen , title =. Journal of Statistical Planning and Inference , volume =. 2012 , pages =

    2012

  86. [94]

    The Annals of Statistics , volume =

    Bunea, Florentina , title =. The Annals of Statistics , volume =. 2004 , pages =

    2004

  87. [95]

    Magnus, and Owen Powell, and Patricia Pr

    Jan R. Magnus, and Owen Powell, and Patricia Pr. A comparison of two model averaging techniques with an application to growth empirics , journal =. 2010 , pages =

    2010

  88. [96]

    Wegkamp , title =

    Bunea, Florentina and Marten H. Wegkamp , title =. The Canadian Journal of Statistics , volume =. 2004 , pages =

    2004

  89. [97]

    Econometric Theory , volume =

    Chen,Songnian and Khan,Shakeeb , title =. Econometric Theory , volume =. 2001 , pages =

    2001

  90. [98]

    The Annals of Statistics , volume =

    Huiliang Xie and Jian Huang , title =. The Annals of Statistics , volume =. 2009 , pages =

    2009

  91. [99]

    Econometric Theory , volume =

    Lian,Heng and Liang,Hua , title =. Econometric Theory , volume =. 2013 , pages =

    2013

  92. [100]

    Journal of the Royal Statistical Society

    Editorial: Model Selection and Efficiency---is `Which Model ...?' the Right Question? , author =. Journal of the Royal Statistical Society. Series A (Statistics in Society) , volume =

  93. [101]

    Econometric Reviews , volume=

    Generalized least squares model averaging , author=. Econometric Reviews , volume=. 2016 , publisher=

    2016

  94. [102]

    Brock, and Steven N

    William A. Brock, and Steven N. Durlauf , year =. Growth Empirics and Reality , journal =

  95. [103]

    and Papageorgiou, Chris and Parmeter, Christopher F

    Henderson, Daniel J. and Papageorgiou, Chris and Parmeter, Christopher F. , title =. The Economic Journal , volume =

  96. [104]

    Journal of Political Economy , volume =

    Determinants of Democracy , author =. Journal of Political Economy , volume =

  97. [105]

    Journal of Economic Growth , volume =

    Democracy and Growth , author =. Journal of Economic Growth , volume =

  98. [106]

    Journal of Econometrics , year=2013, volume=

    Su, Liangjun and Lu, Xun , title=. Journal of Econometrics , year=2013, volume=

    2013

  99. [107]

    and Pan, Jun and Pedersen, Lasse H

    Longstaff, Francis A. and Pan, Jun and Pedersen, Lasse H. and Singleton, Kenneth J. , title =. American Economic Journal: Macroeconomics , year =

  100. [108]

    Econometrica , volume=

    Constructing optimal instruments by first-stage prediction averaging , author=. Econometrica , volume=. 2010 , publisher=

    2010

  101. [109]

    working paper , volume=

    Model Selection and Model Averaging in Nonparametric Instrumental Variables Models , author=. working paper , volume=. 2016 , publisher=

    2016

  102. [110]

    Annals of the Institute of Statistical Mathematics , pages=

    Frequentist model averaging for threshold models , author=. Annals of the Institute of Statistical Mathematics , pages=. 2018 , publisher=

    2018

  103. [111]

    Model Averaging and Its Applications , author=. Ph.D. Thesis , volume=. 2010 , publisher=

    2010

  104. [112]

    Statistics , volume=

    On moments of quadratic forms in non-spherically distributed variables , author=. Statistics , volume=. 1992 , publisher=

    1992

  105. [113]

    1973 , publisher=

    Linear statistical inference and its applications , author=. 1973 , publisher=

    1973

  106. [114]

    Journal of Econometrics , volume=

    Model averaging by jackknife criterion in models with dependent data , author=. Journal of Econometrics , volume=. 2013 , publisher=

    2013

  107. [115]

    Linear or Nonlinear?

    Hao Helen Zhang, and Guang Cheng, and Yufeng Liu , year =. Linear or Nonlinear?

  108. [116]

    Convergence Rates for Parametric Components in a Partly Linear Model , journal = annals, volume =

    Hung Chen , year =. Convergence Rates for Parametric Components in a Partly Linear Model , journal = annals, volume =

  109. [117]

    , title =

    Dungey, Mardi and Martin, Vance L. , title =. Journal of Applied Econometrics , volume =

  110. [118]

    Zongxin Qian, and Wendun Wang, and Kan Ji , title =

  111. [119]

    Forecast Combinations

    Allan Timmermann. Forecast Combinations. Handbook of Economic Forecasting. 2006

    2006

  112. [120]

    D and Bing Li and Francesca Chiaromonte

    Cook, R. D and Bing Li and Francesca Chiaromonte. Envelope models for parsimonious and efficient multivariate linear regression. Statistica Sinica. 2010

    2010

  113. [121]

    Hansen, B. E. , title =. Econometrica , volume =. 2007 , number=

    2007

  114. [122]

    Statistics , volume =

    Rong Zhu and Guohua Zou and Xinyu Zhang , title =. Statistics , volume =. 2018 , publisher =

    2018

  115. [123]

    Maximum Likelihood Estimation of Misspecified Models , volume =

    Halbert White , journal =. Maximum Likelihood Estimation of Misspecified Models , volume =

  116. [124]

    2010 , issn =

    Least squares model averaging by Mallows criterion , journal =. 2010 , issn =

    2010

  117. [125]

    and Okui, R

    Liu, Q. and Okui, R. , title =. The Econometrics Journal , year =

  118. [126]

    Biometrika , year=

    Model Averaging And Weight Choice in Linear Mixed-Effects Models , author=. Biometrika , year=

  119. [127]

    2015 , issn =

    Distribution theory of the least squares averaging estimator , journal =. 2015 , issn =

    2015

  120. [128]

    2015 , issn =

    Consistency of model averaging estimators , journal =. 2015 , issn =

    2015

  121. [129]

    Eck, D. J. and Cook, R. D. , title = ". Biometrika , volume =. 2017 , month =

    2017

  122. [130]

    Annals of the Institute of Statistical Mathematics , year =

    Frequentist model averaging for threshold models , author=. Annals of the Institute of Statistical Mathematics , year =

  123. [131]

    Li , title =

    K.-C. Li , title =. 1987 , volume =

    1987

  124. [132]

    Burnham and David R

    Kenneth P. Burnham and David R. Anderson , title =. Sociological Methods & Research , volume =

  125. [133]

    Raftery, A. E. and Madigan, D. and Hoeting, J. A. , year = 1997, title =

    1997

  126. [134]

    Buckland, S. T. and Burnham, K. P. and Augustin, N. H. , year = 1997, title =

    1997

  127. [135]

    Hoeting, J. A. and Madigan, D. and Raftery, A. E. and Volinsky, C. T. , year = 1999, title =

    1999

  128. [136]

    Dennis , title = "

    Su, Zhihua and Cook, R. Dennis , title = ". Biometrika , volume =. 2011 , month =

    2011

  129. [137]

    Hansen, B. E. and Racine, J. , title =. 2012 , pages =

    2012

  130. [138]

    and Li, K-C

    Ando, T. and Li, K-C. , title =. 2014 , volume =

    2014

  131. [139]

    Proceedings of the National Institute of Sciences of India , volume=

    On the generalised distance in statistics , author=. Proceedings of the National Institute of Sciences of India , volume=

  132. [140]

    Econometric Theory , volume=

    A new study on asymptotic optimality of least squares model averaging , author=. Econometric Theory , volume=. 2021 , publisher=

    2021

  133. [141]

    Econometrica , year =

    Newey, Whitney K , title =. Econometrica , year =

  134. [142]

    arXiv preprint arXiv:2106.09685 , year=

    Lora: Low-rank adaptation of large language models , author=. arXiv preprint arXiv:2106.09685 , year=

    arXiv

  135. [143]

    Statistica Sinica , pages=

    Using input dependent weights for model combination and model selection with multiple sources of data , author=. Statistica Sinica , pages=. 2006 , volume=

    2006

  136. [144]

    Neural Computation , volume=

    Adaptive mixtures of local experts , author=. Neural Computation , volume=. 1991 , publisher=

    1991

  137. [145]

    arXiv preprint arXiv:2106.10199 , year=

    Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models , author=. arXiv preprint arXiv:2106.10199 , year=

  138. [146]

    International conference on machine learning , pages=

    Parameter-efficient transfer learning for NLP , author=. International conference on machine learning , pages=. 2019 , organization=

    2019

  139. [147]

    arXiv preprint arXiv:2110.07577 , year=

    Unipelt: A unified framework for parameter-efficient language model tuning , author=. arXiv preprint arXiv:2110.07577 , year=

  140. [148]

    arXiv preprint arXiv:2307.07726 , year=

    Towards Optimal Neural Networks: the Role of Sample Splitting in Hyperparameter Selection , author=. arXiv preprint arXiv:2307.07726 , year=

  141. [149]

    Working Notes of CLEF 2024 - Conference and Labs of the Evaluation Forum , editor=

    Overview of the Multilingual Text Detoxification Task at PAN 2024 , author=. Working Notes of CLEF 2024 - Conference and Labs of the Evaluation Forum , editor=. 2024 , organization=

    2024

  142. [150]

    arXiv preprint arXiv:1909.10351 , year=

    Tinybert: Distilling bert for natural language understanding , author=. arXiv preprint arXiv:1909.10351 , year=

    1909

  143. [151]

    arXiv preprint arXiv:2303.15647 , year=

    Scaling down to scale up: A guide to parameter-efficient fine-tuning , author=. arXiv preprint arXiv:2303.15647 , year=

  144. [152]

    Journal of Machine Learning Research , volume=

    Optimal parameter-transfer learning by semiparametric model averaging , author=. Journal of Machine Learning Research , volume=

  145. [153]

    2009 , publisher=

    Learning multiple layers of features from tiny images , author=. 2009 , publisher=

    2009

  146. [154]

    Journal of the American Statistical Association , volume=

    Unified optimal model averaging with a general loss function based on cross-validation , author=. Journal of the American Statistical Association , volume=. 2025 , publisher=

    2025

  147. [155]

    arXiv preprint arXiv:2409.03454 , year=

    How much data is enough data? Fine-tuning large language models for in-house translation: Performance evaluation across multiple dataset sizes , author=. arXiv preprint arXiv:2409.03454 , year=

  148. [156]

    arXiv preprint arXiv:2403.07816 , year=

    Branch-train-mix: Mixing expert llms into a mixture-of-experts llm , author=. arXiv preprint arXiv:2403.07816 , year=

  149. [157]

    Journal of Econometrics , volume=

    Generalized aggregation of misspecified models: With an application to asset pricing , author=. Journal of Econometrics , volume=. 2021 , publisher=

    2021

  150. [158]

    Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=

    Model selection principles in misspecified models , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , volume=. 2014 , publisher=

    2014

  151. [159]

    Econometrica , pages=

    Maximum likelihood estimation of misspecified models , author=. Econometrica , pages=. 1982 , number=

    1982

  152. [160]

    1999 , publisher=

    Neural Network Learning: Theoretical Foundations , author=. 1999 , publisher=

    1999

  153. [161]

    The Annals of Statistics , volume=

    Deep nonparametric regression on approximate manifolds: Nonasymptotic error bounds with polynomial prefactors , author=. The Annals of Statistics , volume=. 2023 , publisher=

    2023

  154. [162]

    arXiv preprint arXiv:2012.13255 , year=

    Intrinsic dimensionality explains the effectiveness of language model fine-tuning , author=. arXiv preprint arXiv:2012.13255 , year=

    2012

  155. [163]

    Advances in Neural Information Processing Systems , volume=

    A theoretical analysis of fine-tuning with linear teachers , author=. Advances in Neural Information Processing Systems , volume=

  156. [164]

    arXiv preprint arXiv:2310.19698 , year=

    When do prompting and prefix-tuning work? a theory of capabilities and limitations , author=. arXiv preprint arXiv:2310.19698 , year=

  157. [165]

    Advances in Neural Information Processing Systems , volume=

    Universality and limitations of prompt tuning , author=. Advances in Neural Information Processing Systems , volume=

  158. [166]

    Journal of the Operational Research Society , volume=

    The combination of forecasts , author=. Journal of the Operational Research Society , volume=. 1969 , publisher=

    1969

  159. [167]

    International Journal of Forecasting , volume=

    Adaptively aggregated forecast for exponential family panel model , author=. International Journal of Forecasting , volume=. 2025 , publisher=

    2025

  160. [168]

    Journal of Business & Economic Statistics , volume=

    Model averaging for prediction with fragmentary data , author=. Journal of Business & Economic Statistics , volume=. 2019 , publisher=

    2019

  161. [169]

    Journal of the American Statistical Association , volume=

    On optimality of Mallows model averaging , author=. Journal of the American Statistical Association , volume=. 2025 , publisher=

    2025

  162. [170]

    Model averaging prediction by

    Zhang, Xinyu and Liu, Chu-An , journal=. Model averaging prediction by. 2023 , publisher=

    2023

  163. [171]

    Le and Geoffrey E

    Noam Shazeer and Azalia Mirhoseini and Krzysztof Maziarz and Andy Davis and Quoc V. Le and Geoffrey E. Hinton and Jeff Dean , title =. 5th International Conference on Learning Representations,

  164. [172]

    Journal of Machine Learning Research , volume=

    Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity , author=. Journal of Machine Learning Research , volume=

  165. [173]

    arXiv preprint arXiv:2401.06066 , year=

    Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models , author=. arXiv preprint arXiv:2401.06066 , year=

    arXiv

  166. [174]

    IEEE Transactions on Knowledge and Data Engineering , year=

    A survey on mixture of experts in large language models , author=. IEEE Transactions on Knowledge and Data Engineering , year=

  167. [175]

    Econometric Theory , volume=

    Localized model selection for regression , author=. Econometric Theory , volume=. 2008 , publisher=

    2008

  168. [176]

    Journal of the Royal Statistical Society Series B: Statistical Methodology , pages=

    Prediction sets for high-dimensional mixture of experts models , author=. Journal of the Royal Statistical Society Series B: Statistical Methodology , pages=. 2025 , publisher=

    2025

  169. [177]

    arXiv preprint arXiv:2410.11222 , year=

    Quadratic gating functions in mixture of experts: A statistical insight , author=. arXiv preprint arXiv:2410.11222 , year=

  170. [178]

    arXiv preprint arXiv:2006.16668 , year=

    Gshard: Scaling giant models with conditional computation and automatic sharding , author=. arXiv preprint arXiv:2006.16668 , year=

    2006 arXiv

  171. [179]

    arXiv preprint arXiv:2104.08691 , year=

    The power of scale for parameter-efficient prompt tuning , author=. arXiv preprint arXiv:2104.08691 , year=

    arXiv

  172. [180]

    arXiv preprint arXiv:2101.00190 , year=

    Prefix-tuning: Optimizing continuous prompts for generation , author=. arXiv preprint arXiv:2101.00190 , year=

    arXiv

  173. [181]

    arXiv preprint arXiv:2410.02200 , year=

    Revisiting prefix-tuning: Statistical benefits of reparameterization among prompts , author=. arXiv preprint arXiv:2410.02200 , year=

  174. [182]

    2002 , publisher=

    A Distribution-free Theory of Nonparametric Regression , author=. 2002 , publisher=

    2002

  175. [183]

    2025 , volume =

    Foundations and Trends in Statistics , title =. 2025 , volume =

    2025

  176. [184]

    Journal of Machine Learning Research , volume=

    Nearly-tight VC-dimension and pseudodimension bounds for piecewise linear neural networks , author=. Journal of Machine Learning Research , volume=

  177. [185]

    INFORMS Journal on Computing , volume=

    Reducing simulation input-model risk via input model averaging , author=. INFORMS Journal on Computing , volume=. 2021 , publisher=

    2021

  178. [186]

    Journal of the American Statistical Association , volume=

    Optimal model averaging estimation for generalized linear models and generalized linear mixed-effects models , author=. Journal of the American Statistical Association , volume=. 2016 , publisher=

    2016

  179. [187]

    Learning Theory and Kernel Machines: 16th Annual Conference on Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003, Washington, DC, USA, August 24-27, 2003

    Optimal rates of aggregation , author=. Learning Theory and Kernel Machines: 16th Annual Conference on Learning Theory and 7th Kernel Workshop, COLT/Kernel 2003, Washington, DC, USA, August 24-27, 2003. Proceedings , pages=. 2003 , organization=

    2003

  180. [188]

    Journal of Machine Learning Research , volume=

    Adaptive minimax regression estimation over sparse lq-hulls , author=. Journal of Machine Learning Research , volume=. 2014 , publisher=

    2014

  181. [189]

    Bernoulli , volume=

    Aggregating regression procedures to improve performance , author=. Bernoulli , volume=. 2004 , publisher=

    2004

  182. [190]

    Journal of Econometrics , volume=

    On improvability of model selection by model averaging , author=. Journal of Econometrics , volume=. 2022 , publisher=

    2022

  183. [191]

    Carroll , title =

    Xinyu Zhang and Guohua Zou and Hua Liang and Raymond J. Carroll , title =. Journal of the American Statistical Association , volume =. 2020 , publisher =

    2020

  184. [192]

    Journal of the American Statistical Association , volume=

    Prediction via orthogonalized model mixing , author=. Journal of the American Statistical Association , volume=. 1996 , publisher=

    1996

  185. [193]

    Nature Machine Intelligence , volume=

    Parameter-efficient fine-tuning of large-scale pre-trained language models , author=. Nature Machine Intelligence , volume=. 2023 , publisher=

    2023

  186. [194]

    The Annals of Statistics , volume =

    Juditsky, Anatoli and Nemirovski, Arkadii , title =. The Annals of Statistics , volume =

  187. [195]

    Journal of Multivariate Analysis , volume=

    Combining different procedures for adaptive regression , author=. Journal of Multivariate Analysis , volume=. 2000 , publisher=

    2000

  188. [196]

    Mixing Strategies for Density Estimation , urldate =

    Yuhong Yang , journal =. Mixing Strategies for Density Estimation , urldate =

  189. [197]

    1997 , journal=

    The mixture approach to universal model selection , author=. 1997 , journal=

    1997

  190. [198]

    1999 , journal=

    `Universal' aggregation rules with exact bias bounds , author=. 1999 , journal=

    1999

  191. [199]

    Journal of the American Statistical Association , volume=

    Frequentist model average estimators , author=. Journal of the American Statistical Association , volume=. 2003 , publisher=

    2003

  192. [200]

    Dalalyan and Joseph Salmon , title =

    Arnak S. Dalalyan and Joseph Salmon , title =. The Annals of Statistics , number =

  193. [201]

    2004 , publisher=

    Statistical Learning Theory and Stochastic Optimization , author=. 2004 , publisher=

    2004

  194. [202]

    Information and Inference: A Journal of the IMA , volume =

    Chen, Minshuo and Jiang, Haoming and Liao, Wenjing and Zhao, Tuo , title =. Information and Inference: A Journal of the IMA , volume =. 2022 , month =

    2022

Pith tools

Reviewed May 14, 2026 · model on record in the stance chip above.