Outliers in meta-analysis: an asymmetric trimmed-mean approach

Rose Baker

arxiv: 1907.07015 · v1 · pith:ZM32PMUYnew · submitted 2019-07-16 · 📊 stat.AP

Outliers in meta-analysis: an asymmetric trimmed-mean approach

Rose Baker This is my paper

Pith reviewed 2026-05-24 20:35 UTC · model grok-4.3

classification 📊 stat.AP

keywords meta-analysisoutlierstrimmed meanbootstraprobust statisticsdown-weighting

0 comments

The pith

A modified adaptive asymmetric trimmed mean down-weights outlying studies in meta-analysis without parametric assumptions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper adapts the adaptive asymmetric trimmed mean estimator to meta-analysis for handling outliers by down-weighting entire studies. This is achieved through a modified bootstrap procedure that preserves the method's properties. Analyses of standard datasets show agreement with other outlier detection approaches, while Monte Carlo simulations indicate minimal down-weighting when no outliers are present. The method is conceptually simple and avoids assumptions about the distribution of outliers.

Core claim

The adaptive asymmetric trimmed mean can be modified for meta-analysis by down-weighting studies using a modified bootstrap, providing a way to deal with outlying results that does not rely on parametric assumptions about the outliers.

What carries the argument

The adaptive asymmetric trimmed mean, adapted with a modified bootstrap to down-weight studies instead of single observations.

If this is right

It down-weights outliers in agreement with other methods on well-travelled datasets.
Monte-Carlo studies show it does not appreciably down-weight studies when outliers are absent.
It remains conceptually simple and makes no parametric assumptions about the outliers.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

This could extend to other statistical contexts where robust estimation of location is needed without distributional assumptions.
Researchers in fields using meta-analysis might find it useful for sensitivity analysis to extreme study results.
Further Monte Carlo experiments could test its performance under different outlier magnitudes or numbers of studies.

Load-bearing premise

That modifying the adaptive asymmetric trimmed mean with a bootstrap procedure allows it to down-weight studies in meta-analysis while retaining its outlier-handling capabilities.

What would settle it

Observing in Monte-Carlo simulations that the method down-weights studies substantially even in the absence of outliers, or that it fails to down-weight known outliers in real meta-analysis datasets.

read the original abstract

The adaptive asymmetric trimmed mean is a known way of estimating central location, usually in conjunction with the bootstrap. It is here modified and applied to meta-analysis, as a way of dealing with outlying results by down-weighting the corresponding studies. This requires a modified bootstrap and a method of down-weighting studies, as opposed to removing single observations. This methodology is shown in analysis of some well-travelled datasets to down-weight outliers in agreement with other methods, and Monte-Carlo studies show that it does does not appreciably down-weight studies when outliers are absent. Conceptually simple, it does not make parametric assumptions about the outliers.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

A modest, practical tweak to an existing robust estimator for down-weighting outlier studies in meta-analysis, with simulations and examples that line up with the claims.

read the letter

The core point is that this paper takes the adaptive asymmetric trimmed mean, already known for location estimation, and adapts it specifically for meta-analysis by down-weighting entire studies rather than single observations, using a modified bootstrap. It checks out on a few standard datasets by agreeing with other outlier methods and in Monte Carlo runs by mostly leaving studies alone when no outliers are present. That is the main new piece: the study-level application and the bootstrap adjustment for this context. The work is conceptually straightforward and avoids parametric assumptions about what the outliers look like, which is a plus for applied users in medicine or social sciences. The real-data checks and the null-case simulations are the parts that actually support the claims, and they appear consistent with the stated goals. The limitation is that the novelty sits mostly in the targeted modification rather than new theory or large-scale validation; the bootstrap details would need close scrutiny in review to confirm they preserve the estimator's properties. No circularity or hidden assumptions jump out from what is described. This is the kind of paper that could be useful to practitioners who already do meta-analysis and want a simple non-parametric option, but it is not something that changes the broader literature. A serious editor should send it for peer review because the evidence matches the scope of the contribution and the method is reproducible enough to be tested further.

Referee Report

0 major / 4 minor

Summary. The paper modifies the adaptive asymmetric trimmed mean estimator for meta-analysis by introducing study-level down-weighting (instead of observation removal) together with a tailored bootstrap procedure. It reports that the resulting method down-weights outliers on several well-known datasets in agreement with existing robust meta-analytic approaches, while Monte Carlo experiments show that studies are not appreciably down-weighted when no outliers are present. The approach is presented as conceptually simple and free of parametric assumptions on the outlier distribution.

Significance. If the modification preserves the original estimator's outlier-handling properties, the method supplies a non-parametric, easy-to-implement tool for robust meta-analysis that avoids strong distributional assumptions. The real-data agreement and null-case simulation results provide direct empirical support for the central claim.

minor comments (4)

Abstract: the phrase 'it does does not appreciably' contains a repeated word; this should be corrected.
The manuscript should supply an explicit algorithmic description or pseudocode for the study-level weight calculation and the modified bootstrap resampling scheme (currently described only in prose) so that readers can reproduce the procedure without ambiguity.
Section describing the Monte Carlo design should state the exact number of replications, the heterogeneity parameters, and the precise criterion used to declare that a study is 'not appreciably down-weighted'.
The real-data examples would benefit from a table that reports, for each study, both the original weight and the down-weighted value together with the corresponding outlier diagnostics from the comparator methods.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive assessment of the manuscript, the accurate summary of the proposed method, and the recommendation for minor revision. No specific major comments were provided in the report.

Circularity Check

0 steps flagged

No significant circularity detected

full rationale

The paper modifies a pre-existing adaptive asymmetric trimmed mean estimator (cited as known) for meta-analysis by introducing study-level down-weighting and a tailored bootstrap; these modifications are then evaluated on independent real datasets for agreement with other robust methods and on Monte-Carlo simulations under the explicit null of no outliers. No derivation step, equation, or central claim reduces by construction to a fitted parameter, self-citation, or input data; the validation procedures are external to the method definition itself.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The approach rests on the prior existence of the adaptive asymmetric trimmed mean and bootstrap methods. The main addition is the adaptation to study-level weighting in meta-analysis. No free parameters, invented entities, or ad-hoc axioms are mentioned in the abstract.

axioms (1)

domain assumption The adaptive asymmetric trimmed mean can be adapted from single observations to entire studies in a meta-analysis setting while retaining its properties via a modified bootstrap.
This is the central modification required for the method to apply to meta-analysis.

pith-pipeline@v0.9.0 · 5617 in / 1324 out tokens · 23126 ms · 2026-05-24T20:35:14.896644+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

16 extracted references · 16 canonical work pages

[1]

Lahiani, A

Aleem, A. Lahiani, A. (2011). Estimation and evaluation of core inﬂ ation measures, Applied Economics, 43, 3619-3629

work page 2011
[2]

Baker R. D. and Jackson D. (2008). A new approach to outliers in meta- analysis, Healthcare Management Science, 11 121-131

work page 2008
[3]

Baker R. D. and Jackson D. (2016). New models for describing ou tliers in meta-analysis, Research Synthesis Methods, 7, 314-328

work page 2016
[4]

Beath, K. (2014). A ﬁnite mixture method for outlier detection a nd robustness in meta-analysis, Research Synthesis Methods 5 (4), 285- 293

work page 2014
[5]

DerSimonian R., Laird N. (1986). Meta-analysis in clinical trials. Con - trolled Clinical Trials 7 177-188

work page 1986
[6]

and Tibshirani, R

Efron, B. and Tibshirani, R. J. (1993). An Introduction to the B ootstrap, Chapman and Hall, London

work page 1993
[7]

Gumedze F. N. and Jackson D. (2011). A random eﬀects varianc e shift model for detecting and accommodating outliers in meta-analysis, B MC Medical Research Methodology, 11:19

work page 2011
[8]

and Sinha, B

Hartung, J., Knapp, G. and Sinha, B. K. (2008). Statistical met a- analysis with applications, Wiley, New York

work page 2008
[9]

Higgins, J. P. T. and Spiegelhalter, D. J. (2002). Being sceptical about meta-analyses: a Bayesian perspective on magnesium trials in myoca r- dial infarction, International Journal of Epidemiology, 31, 96-104. 12

work page 2002
[10]

Julious S. A. and Whitehead, A. (2011). Investigating the assu mption of homogeneity of treatment eﬀects in clinical studies with applicatio n to meta-analysis, Pharmaceutical Statistics 11, 49-56

work page 2011
[11]

and Hodges, J

Lin, L., Chu, H. and Hodges, J. S. (2017). Alternative measure s of between-study heterogeneity in meta-analysis: reducing the impa ct of outlying studies, Biometrics 73, 156-166

work page 2017
[12]

and Romano, J

L´ eger, C. and Romano, J. P. (1990). Bootstrap adaptive es timation: the trimmed mean example, The Canadian Journal of Statistics 18, 297-314

work page 1990
[13]

Mavridis, D., Moustaki, I., Wall, M., Salanti, G. (2017). Detecting o utly- ing studies in meta-regression models using a forward search algorit hm, Research Synthesis Methods, 8, 199-211

work page 2017
[14]

and Onghena, P

Van Den Noortgate, W. and Onghena, P. (2005). Parametric a nd non- parametric bootstrap methods for meta-analysis, Behavior Rese arch methods, 37, 11-22

work page 2005
[15]

Sharma, N. K. and Bicchal, M. (2015). Measuring core inﬂation in In- dia: An asymmetric trimmed mean approach, Cogent Economics and Finance, 3, article number 1014252

work page 2015
[16]

and Cheung, M

Viechtbauer, W. and Cheung, M. W.-L. (2010). Outlier and inﬂue nce diagnostics for meta-analysis, Research Synthesis Methods 1 (2), 112- 125. 13 Appendix: code Currently, the code is a fortran95 program that reads a ‘control’ ﬁle of com- mands, and the dataﬁle. It uses the NAG library for random-numbe r genera- tion and for computing the normal distributi...

work page 2010

[1] [1]

Lahiani, A

Aleem, A. Lahiani, A. (2011). Estimation and evaluation of core inﬂ ation measures, Applied Economics, 43, 3619-3629

work page 2011

[2] [2]

Baker R. D. and Jackson D. (2008). A new approach to outliers in meta- analysis, Healthcare Management Science, 11 121-131

work page 2008

[3] [3]

Baker R. D. and Jackson D. (2016). New models for describing ou tliers in meta-analysis, Research Synthesis Methods, 7, 314-328

work page 2016

[4] [4]

Beath, K. (2014). A ﬁnite mixture method for outlier detection a nd robustness in meta-analysis, Research Synthesis Methods 5 (4), 285- 293

work page 2014

[5] [5]

DerSimonian R., Laird N. (1986). Meta-analysis in clinical trials. Con - trolled Clinical Trials 7 177-188

work page 1986

[6] [6]

and Tibshirani, R

Efron, B. and Tibshirani, R. J. (1993). An Introduction to the B ootstrap, Chapman and Hall, London

work page 1993

[7] [7]

Gumedze F. N. and Jackson D. (2011). A random eﬀects varianc e shift model for detecting and accommodating outliers in meta-analysis, B MC Medical Research Methodology, 11:19

work page 2011

[8] [8]

and Sinha, B

Hartung, J., Knapp, G. and Sinha, B. K. (2008). Statistical met a- analysis with applications, Wiley, New York

work page 2008

[9] [9]

Higgins, J. P. T. and Spiegelhalter, D. J. (2002). Being sceptical about meta-analyses: a Bayesian perspective on magnesium trials in myoca r- dial infarction, International Journal of Epidemiology, 31, 96-104. 12

work page 2002

[10] [10]

Julious S. A. and Whitehead, A. (2011). Investigating the assu mption of homogeneity of treatment eﬀects in clinical studies with applicatio n to meta-analysis, Pharmaceutical Statistics 11, 49-56

work page 2011

[11] [11]

and Hodges, J

Lin, L., Chu, H. and Hodges, J. S. (2017). Alternative measure s of between-study heterogeneity in meta-analysis: reducing the impa ct of outlying studies, Biometrics 73, 156-166

work page 2017

[12] [12]

and Romano, J

L´ eger, C. and Romano, J. P. (1990). Bootstrap adaptive es timation: the trimmed mean example, The Canadian Journal of Statistics 18, 297-314

work page 1990

[13] [13]

Mavridis, D., Moustaki, I., Wall, M., Salanti, G. (2017). Detecting o utly- ing studies in meta-regression models using a forward search algorit hm, Research Synthesis Methods, 8, 199-211

work page 2017

[14] [14]

and Onghena, P

Van Den Noortgate, W. and Onghena, P. (2005). Parametric a nd non- parametric bootstrap methods for meta-analysis, Behavior Rese arch methods, 37, 11-22

work page 2005

[15] [15]

Sharma, N. K. and Bicchal, M. (2015). Measuring core inﬂation in In- dia: An asymmetric trimmed mean approach, Cogent Economics and Finance, 3, article number 1014252

work page 2015

[16] [16]

and Cheung, M

Viechtbauer, W. and Cheung, M. W.-L. (2010). Outlier and inﬂue nce diagnostics for meta-analysis, Research Synthesis Methods 1 (2), 112- 125. 13 Appendix: code Currently, the code is a fortran95 program that reads a ‘control’ ﬁle of com- mands, and the dataﬁle. It uses the NAG library for random-numbe r genera- tion and for computing the normal distributi...

work page 2010