Pith. sign in

REVIEW 3 major objections 5 minor 33 references

Empirical likelihood meta analysis with publication bias correction under Copas-like selection model

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a full likelihood formed by combining the Copas conditional likelihood with an empirical likelihood for study standard errors gives jointly normal estimators and asymptotically chi-square likelihood ratios, and that…

desk verdict Serious EL-based full-likelihood method for Copas selection, but the theory covers the uncomputed full MLE while the implementation uses plug-in gammas; the coverage gains need interval widths and a proof for the implemented procedure. read the letter →

arxiv 2507.13615 v1 pith:IQML7PGU submitted 2025-07-18 stat.ME

Mengke Li , Yukun Liu , Pengfei Li , Jing Qin This is my paper · ORCID
classification stat.ME MSC 62F1262G05
keywords meta-analysispublicationbiasCopasselectionmodelempiricallikelihoodfullratioconfidenceintervalsbetween-studyheterogeneitytotalnumberofstudies
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Meta-analysis combines published studies, but publication bias and between-study heterogeneity can distort conclusions. The paper develops a full-likelihood method for a Copas-style selection model by combining the usual conditional likelihood with an empirical likelihood for the marginal distribution of study standard errors. It claims that the resulting maximum likelihood estimators are jointly asymptotically normal and that the full likelihood ratio is asymptotically chi-square, enabling likelihood-ratio confidence intervals for the overall effect size and for the total number of studies. In simulations, these intervals have noticeably more accurate coverage than the conditional-likelihood Wald intervals, and the point estimators usually have smaller mean squared errors. The paper also shows that all underlying parameters are identifiable whenever publication bias is present ($\rho\neq 0$).

What carries the argument

The load-bearing object is the profile semiparametric full log-likelihood, obtained by replacing the unknown marginal distribution of the study standard errors $s_i^*$ with Owen empirical-likelihood weights $p_i=n^{-1}[1+\lambda\{\Phi(\gamma_1+\gamma_2/s_i)-\alpha\}]^{-1}$ satisfying $\sum_i p_i=1$ and $\sum_i p_i\Phi(\gamma_1+\gamma_2/s_i)=\alpha$. With $v_i(\gamma)=(\gamma_1+\gamma_2/s_i+\rho s_i(\theta_i-\theta)/(\tau^2+s_i^2))/\sqrt{1-\rho^2 s_i^2/(\tau^2+s_i^2)}$, the profile log-likelihood is $$\ell(N,\$\alpha$,\gamma)=\log\binom{N}{n}+(N-n)\log(1-\$\alpha$)+\sum_i\left[\log\Phi(v_i(\gamma))-\frac12\log(\$tau^{2}$+$s_i^{2}$)-\frac{(\theta_i-\$\theta$)^2}{2(\$tau^{2}$+$s_i^{2}$)}\right]-\sum_i\log[1+\$\lambda$\{\Phi(\gamma_1+\gamma_2/s_i)-\$\alpha$\}].$$ This turns the conditional Copas likelihood into a full likelihood that also uses information in the observed number of studies $n$ and in the marginal distribution of $s^*$. Theorem 1 establishes joint asymptotic normality of the maximizers and a $\chi^2_7$ limit for the full likelihood ratio; Proposition 1 shows the asymptotic covariance equals that of the conditional likelihood.

What would settle it

Generate simulated meta-analyses from the model with known parameters, but choose settings where the conditional EM estimates of $\gamma_1$ and $\gamma_2$ are visibly far from the full likelihood maximizers; compute the implemented two-step estimator and check whether 95% likelihood-ratio intervals for $\theta$ and $N$ still cover at about 95%. A large drop in coverage when the plugged-in $\gamma_{12}$ differs materially from the true MLE would refute the claim that the implemented estimator inherits Theorem 1's chi-square limit.

Watch

Extended reading notes

Core claim

Under the Copas-like selection model of Ning et al. (2017), the paper proposes a full semiparametric likelihood that joins the conditional likelihood for the observed effect estimates with Owen's empirical likelihood for the unknown marginal distribution of study standard errors. The paper proves that, provided publication bias exists ($\rho\neq 0$) and mild regularity conditions hold, the full likelihood maximizers of $(N,\alpha,\gamma)$ are jointly asymptotically normal and the full likelihood ratio statistic converges to a central $\chi^2_7$ distribution. This yields likelihood-ratio confidence intervals for $\theta$, $\tau$, $\rho$, $\alpha$, and $N$ with asymptotically correct coverage, and consistent estimators of the marginal distributions of standard errors and effect sizes. Proposition 1 shows that the asymptotic variances of the full and conditional estimators of $N$ and $\gamma$ are identical, so the full likelihood offers no asymptotic point-estimation efficiency gain; its advantage lies in likelihood-ratio inference and in avoiding unstable inverse-probability weights. Simulation results in the paper report smaller mean squared errors and, especially, more accurate coverage probabilities for the full likelihood intervals than for conditional-likelihood Wald intervals, and a real data example illustrates that the two methods can differ on whether publication bias is detected.

Load-bearing premise

The method as actually computed fixes the two parameters that determine how strongly a study is selected for publication to values obtained from a separate fitting procedure, and the paper assumes without proof that this substitution does not change the asymptotic results or interval coverage.

Editorial extensions

If this is right

  • Under the assumed model with $\rho\neq 0$, the full likelihood approach gives consistent, jointly normal estimators of $\theta$, $\tau$, $\rho$, $\alpha$, and $N$, and likelihood-ratio statistics with asymptotic chi-square limits, so confidence intervals can be built without estimating variances.
  • Because the empirical likelihood weights prevent extreme small probabilities, the estimated marginal distributions of study standard errors and effect sizes avoid the instability of inverse-probability-weighting estimators.
  • The Proposition 1 result implies that the full likelihood does not improve asymptotic point-estimation efficiency over the conditional likelihood; its benefit is in likelihood-ratio inference rather than in smaller asymptotic variance.
  • In the premature-birth example, the full likelihood ratio interval for $\rho$ excludes zero while the conditional Wald interval contains zero, so the method can change the conclusion about whether publication bias is present.
  • The likelihood-ratio interval for $N$ respects the constraint $N\ge n$, avoiding Wald intervals whose lower bounds fall below the number of observed studies.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the equality of asymptotic variances in Proposition 1 suggests the full likelihood's practical value may generalize across biased-sample problems: point estimation is not improved asymptotically, but likelihood-ratio inference avoids variance estimation and naturally respects parameter constraints such as $N\ge n$.
  • Editorial inference: because the flatness in $(\gamma_1,\gamma_2)$ is the computational bottleneck, profiling the full likelihood over these two parameters or calibrating the two-step likelihood ratio by bootstrap could yield an implemented estimator for which the stated chi-square theory literally holds, rather than the current fixed-$\tilde\gamma_{12}$ version.
  • Editorial inference: the observed superiority of the chi-square approximation over the normal approximation to the Wald statistic is a finite-sample phenomenon; a natural testable extension is whether it persists under misspecification of the selection equation or of the random-effects distribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a full likelihood approach for meta-analysis under the Copas-like selection model of Ning et al. (2017). It combines the conditional likelihood for the published effect sizes with an empirical likelihood for the marginal distribution of the study-specific standard errors, and treats the total number of studies N as an unknown binomial parameter. The main theoretical result (Theorem 1) claims joint asymptotic normality of the full likelihood estimators of (N, alpha, gamma) and a central chi-square limit for the likelihood ratio statistic; Theorem 2 and Proposition 1 give the corresponding conditional-likelihood results and establish asymptotic equivalence of the two point estimators. Simulations compare the proposed estimators and likelihood ratio intervals with conditional-likelihood estimators and Wald intervals, and a real data example is analyzed. The discussion acknowledges that the implemented algorithm fixes gamma1 and gamma2 at conditional maximum likelihood estimates rather than maximizing the full likelihood over all parameters.

Significance. If the chi-square calibration of the implemented likelihood ratio statistic could be justified, the paper would provide a practically useful route to confidence intervals for N, theta, tau, and rho that avoids the instability of inverse-probability-weighting and the absurd Wald lower bounds for N. The empirical-likelihood formulation of the marginal distribution of the standard errors is natural, the profile likelihood derivation is coherent, the simulation study is extensive, and the QQ-plot diagnostic is a useful addition. The main limitation is that the asymptotic theory does not cover the two-step estimator actually implemented, and Proposition 1 shows that there is no asymptotic efficiency gain over conditional likelihood, so the contribution rests on finite-sample and interval-calibration properties that are currently supported only by simulation.

major comments (3)
  1. [Section 2.5 and Section 5] The algorithm in Section 2.5 fixes gamma12 at the conditional MLE from Ning et al. (2017) and then maximizes over the remaining parameters, but Theorem 1 is stated for the unconstrained maximizer of ell(N, alpha, gamma). The Discussion in Section 5 concedes that the resulting parameter estimates are 'in essence different from the true maximum likelihood estimates.' Replacing a subset of nuisance parameters by a root-n consistent estimator in a profile likelihood does not in general preserve the chi-square limit of the likelihood ratio statistic unless an orthogonality or flatness condition holds; the flat-plateau statement is a heuristic and no verification is provided. Consequently, the asymptotic justification for the likelihood ratio intervals in Tables 1 and 2 and the QQ-plots in Figure 1 does not formally apply to the estimator actually computed. The authors should either prove the relevant profile-likelihood result for the two-step estimator, verify the negligibility condition empirically by comparing the two-step and full maximizers on simulated data, or explicitly reframe the theoretical claims and provide an alternative calibration for the implemented procedure.
  2. [Section 3.2 and Proposition 1] The finite-sample coverage comparisons compare likelihood ratio intervals based on the proposed full-likelihood procedure with Wald intervals based on the conditional-likelihood procedure. Since Proposition 1 shows that the two point estimators are asymptotically equivalent, the reported coverage improvements may be due to the use of likelihood ratio calibration rather than to the full-likelihood estimator itself. To support the attribution in the abstract and introduction, the authors should add a comparison with likelihood-ratio-type intervals based on the conditional likelihood or with Wald intervals computed from the full-likelihood estimates; otherwise the conclusion should be softened to claim an improvement of the full-likelihood LR interval over the conditional-likelihood Wald interval as a complete procedure.
  3. [Theorem 1 and supplementary conditions] Theorem 1 is stated under Conditions C1 and C2, which are described only as being in the supplementary material, and the proof is deferred as well. The main text should at least state these conditions and comment on whether the simulation designs satisfy them, particularly the positive-definiteness of Omega and the moment conditions needed for the empirical likelihood weights. Without this, the reader cannot assess whether the simulation settings fall within the theoretical regime claimed by Theorem 1.
minor comments (5)
  1. [Section 4] The sentence 'For point estimation of N, the full likelihood and conditional likelihood estimates are 23 and 16, respectively' is reversed relative to Table 3, which reports CL = 23 and FL = 16.
  2. [Table 3 caption] The caption says 'Meta analysis results of the lung cancer data and the premature birth data,' but the text and the rest of Section 4 analyze only the premature birth data.
  3. [Section 2.1] The quantity s_i is introduced as 'the estimated standard variance'; given its role as the standard error in models (1) and (2), the term 'standard error' or 'standard deviation' would be clearer and would avoid confusion with s_i^2 as a variance.
  4. [Section 5 and references] The phrase 'To then end' should be 'To this end,' and the reference 'Contrilled Clinical Trials' should be 'Controlled Clinical Trials.'
  5. [Section 2.4] The statement that the proof of Theorem 1 implies a chi-square limit for likelihood ratio statistics for any subvector of (N, alpha, gamma) is asserted informally; a formal statement of this profile likelihood result would strengthen the interval-construction claims.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: the full likelihood, EL constraint, and chi-square limit are derived from the model and external Ning et al. (2017) EM implementation; self-citations are contextual, not load-bearing.

full rationale

The derivation chain is self-contained. The full likelihood in Equation (3) is obtained by factorizing the joint distribution of observed studies under the Copas-like selection model, with the selection probability terms given in Lemma 2. The empirical likelihood weights are not fitted to a target result: the constraint in Equation (8) follows from alpha = pr(Z_i > 0) = E{Phi(gamma_1 + gamma_2 / s_i^*)}, so the profiling step is a direct consequence of the model. Theorem 1's joint normality and chi-square likelihood-ratio limits are stated under explicit conditions C1 and C2 with proofs deferred to the supplementary material, and Proposition 1 honestly reports that the full likelihood estimators have the same asymptotic variance as the conditional MLEs, which is a negative efficiency result rather than a circular one. The only notable gap is in Section 2.5, where the authors fix gamma_1 and gamma_2 at the conditional MLEs from Ning et al. (2017) because direct maximization is unstable, and the Discussion admits that 'the resulting parameter estimates are in essence different from the true maximum likelihood estimates.' This is a mismatch between the implemented estimator and the estimator analyzed in Theorem 1, and the flat-plateau justification is heuristic, but it is a correctness/robustness concern, not circularity: the plugged-in values come from an external EM algorithm and are not derived from the paper's own chi-square or normality claims. The self-citations to Liu et al. (2017, 2018) about poor Wald lower bounds for N are contextual and do not carry the main argument; the empirical likelihood machinery is attributed to Owen (1990). Therefore no step in the claimed derivation reduces to its own inputs by construction.

Assumptions & free parameters 2 free parameters · 6 assumptions · 0 invented entities

The central claim depends on modeling assumptions inherited from Ning et al. (2017), on the identifiability condition rho != 0, and on the unproven two-step implementation of the full likelihood where gamma1 and gamma2 are fixed at conditional MLEs. No new physical or conceptual entities are introduced beyond the existing latent selection variable Z_i.

free parameters (2)
  • gamma1, gamma2 (selection intercept and slope) = Estimated by Ning et al. (2017) EM algorithm, not by full MLE
    Section 2.5 fixes these to their conditional MLE values to avoid non-convergence. The theoretical MLE results in Theorem 1 do not apply to this two-step estimator, and the effect of this fixing on LR statistics is not derived.
  • cn (EL log* threshold) = n (sample size)
    Section 2.5 sets cn = n in the modified log function, a tuning choice that affects the empirical likelihood approximation in the h3 term.
assumptions (6)
  • standard math Regularity conditions C1 and C2 in the supplementary material
    Invoked for the asymptotic normality of MLEs and the chi-square limit of the likelihood ratio in Theorem 1.
  • domain assumption (theta*_i, s*_i) are IID and independent of each other
    Section 2.1 model setup; central to the likelihood factorization and the biased-sample interpretation.
  • domain assumption Random effects model (1): theta*_i = theta + tau u_i + s*_i epsilon_i with u_i, epsilon_i standard normal and independent
    Equation (1) defines the study effect distribution conditional on s*_i.
  • domain assumption Selection model (2): Z_i = gamma1 + gamma2/s*_i + delta_i, with (epsilon_i, delta_i) bivariate normal with correlation rho, and study published if Z_i > 0
    Equation (2) defines the publication mechanism; this is the Copas-like model inherited from Ning et al. (2017).
  • domain assumption rho != 0 (publication bias exists)
    Lemma 1 shows identifiability only when rho != 0. The paper assumes this throughout; if rho = 0 the model is unidentifiable and the method may behave poorly.
  • ad hoc to paper The effect of fixing gamma1 and gamma2 to conditional MLEs is negligible for LR statistics
    Section 2.5 and the Discussion state this informally, but no proof or bound is provided. This is the load-bearing assumption behind using the implemented two-step estimator as if it were the full MLE.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Empirical likelihood meta analysis with publication bias correction under Copas-like selection model." pith.science (2026). https://pith.science/paper/IQML7PGU

@misc{pith2026250713615,
  author       = {Pith},
  title        = {Pith review of: Empirical likelihood meta analysis with publication bias correction under Copas-like selection model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/IQML7PGU}},
  note         = {Machine review of arXiv:2507.13615}
}
read the original abstract

Meta analysis is commonly-used to synthesize multiple results from individual studies. However, its validation is usually threatened by publication bias and between-study heterogeneity, which can be captured by the Copas selection model. Existing inference methods under this model are all based on conditional likelihood and may not be fully efficient. In this paper, we propose a full likelihood approach to meta analysis by integrating the conditional likelihood and a marginal semi-parametric empirical likelihood under a Copas-like selection model. We show that the maximum likelihood estimators (MLE) of all the underlying parameters have a jointly normal limiting distribution, and the full likelihood ratio follows an asymptotic central chisquare distribution. Our simulation results indicate that compared with the conditional likelihood method, the proposed MLEs have smaller mean squared errors and the full likelihood ratio confidence intervals have more accurate coverage probabilities. A real data example is analyzed to show the advantages of the full likelihood method over the conditional likelihood method.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 33 canonical work pages

  1. [1]

    Carpenter, J, G

    R. Carpenter, J, G. Schwarzer, G. Rucker, and R. Kunstler. Empirical evaluation showed that the copas selection model provided a useful summary in 80\ Journal of Clinical Epidemiology, 62: 0 624--631, 2009

    work page 2009

  2. [2]

    Cooper, L

    H. Cooper, L. V. Hedges, and J. C. Valentine. The handbook of research synthesis and meta-analysis, 2nd edn. Russell Sage Foundation, New York., 2009

    work page 2009

  3. [3]

    J. Copas. What works?: selectivity models and meta-analysis. Journal of the Royal Statistical Society: Series A,, 162: 0 95--109, 1999

    work page 1999

  4. [4]

    Copas and D

    J. Copas and D. Jackson. A bound for publication bias based on the fraction of unpublished studies. Biometrics, 60 0 (1): 0 146--153, 2004

    work page 2004

  5. [5]

    Copas and J

    J. Copas and J. Q. Shi. Meta-analysis, funnel plots and sensitivity analysis. Biostatistics, 1: 0 247--262, 2000

    work page 2000

  6. [6]

    Copas and J .Q

    J. Copas and J .Q. Shi. A sensitivity analysis for publication bias in systematic reviews. Statistical Methods in Medical Research, 10: 0 251--265, 2001

    work page 2001

  7. [7]

    J. B. Copas and H. G. Li. Inference for non-random samples. Journal of the Royal Statistical Society: Series B., 59: 0 55--95, 1997

    work page 1997

  8. [8]

    DerSimonian and N

    R. DerSimonian and N. Laird. Meta-analysis in clinical trials. Contrilled Clinical Trials, 7: 0 177--188, 1986

    work page 1986

Show all 33 references
  1. [9]

    Duval and R

    S. Duval and R. Tweedie. A nonparametric 'trim and fill' method of accounting for publication bias in meta-analysis. Journal of the American Statistical Association, 95: 0 89--98, 2000 a

    2000

  2. [10]

    Duval and R

    S. Duval and R. Tweedie. Trim and fill: A simple funnel-plot based method of testing and adjusting for publication bias in meta-analysis. Biometrics, 56: 0 455--463., 2000 b

    2000

  3. [11]

    and Minder C

    and Smith G .D and Schineider M. and Minder C. Egger, M. Bias in meta-analysis detected by a simple, graphical test. British Medical Journal, 315: 0 629--634, 1997

    1997

  4. [12]

    Egger, G

    M. Egger, G. D. Smith, and D. G. Altman. Systematic reviews in health care: meta-analysis in context, 2nd edn. BMJ, London,, 2001

    2001

  5. [13]

    Fragkos, M

    K .C. Fragkos, M. Tsagris, and C. C. Frangos. Exploring the distribution for the estimator of rosenthal's 'fail-safe' number of unpublished studies in meta-analysis. Communications in Statistics - Theory and Methods, 46 0 (11): 0 5672--5684, 2017

    2017

  6. [14]

    Galbraith

    R. Galbraith. A note on the graphical presentation of estimated odds ratios from several clinical trials. Statistics in Medicine, 7: 0 889--894, 1988

    1988

  7. [15]

    G. V. Glass. Primary, secondary, and meta-analysis of research. Educational Researcher, 5 0 (10): 0 3--8, 1976

    1976

  8. [16]

    L. J. Gleser and I. Olkin. Models for estimating the number of unpublished studies. Statistics in Medicine, 15: 0 2493--2507, 1996

    1996

  9. [17]

    P. Han. Multiply robust estimation in regression analysis with missing data. Journal of the American Statistical Association, 109: 0 1159--1173, 2014

    2014

  10. [18]

    Jackson, R

    D. Jackson, R. Riley, and R. White I. Multivariate meta-analysis: potential and promise. Statistics in Medicine, 30: 0 2481--2498, 2011

    2011

  11. [19]

    Z. C. Jin, X .H. Zhou, and J. He. Statistical method for dealing with publication bias in meta-analysis. Statistics in Medicine, 34: 0 343--360, 2015

    2015

  12. [20]

    Koricheva, J

    J. Koricheva, J. Gurevitch, and K. Mengersen. The handbook of meta-analysis in ecology and evolution. Princeton University Press, Princeton, 2012

    2012

  13. [21]

    Light and D

    R. Light and D. Pillemer. Summing Up: the Science of Reviewing Research. Cambridge, MA: Harvard University Press, 1984

    1984

  14. [22]

    Y. Liu, P. Li, and J. Qin. Maximum empirical likelihood estimation for abundance in a closed population from capture recapture data. Biometrika, 104: 0 527--543, 2017

    2017

  15. [23]

    Y. Liu, Y. Liu, P. Li, and J. Qin. Full likelihood inference for abundance from continuous-time capture-recapture data. Journal of the Royal Statistical Society: Series B, 80 0 (5): 0 995--1014, 2018

    2018

  16. [24]

    T. A. Louis. Finding the observed information matrix when using the em algorithm. Journal of the Royal Statistical Society: Series B, 44: 0 226--233, 1982

    1982

  17. [25]

    Mavridis, A

    D. Mavridis, A. Sutton, A. Cipriani, and G. Salanti. A fully bayesian application of the copas selection model for publication bias extended to network meta-analysis. Statistics in Medicine, 32 0 (1): 0 51--66, 2013

    2013

  18. [26]

    J. Ning, Y. Chen, and J. Piao. Maximum likelihood estimation and em algorithm of copas-like selection model for publication bias correction. Biostatistics, 18 0 (3): 0 495--504, 2017

    2017

  19. [27]

    A. B. Owen. Empirical likelihood ratio confidence regions. Annals of Statistics,, 18: 0 90--120, 1990

    1990

  20. [28]

    file drawer problem

    R. Rosenthal. The "file drawer problem" and tolerance for null results. Psychological Bulletin, 86: 0 638--641, 1979

    1979

  21. [29]

    H. R. Rothstein. Publication bias as a threat to the validity of meta-analytic results. Journal of Experimental Criminology, 4: 0 61--81, 2008

    2008

  22. [30]

    H. R. Rothstein, A .J. Sutto, and M. Borenstein. Publication Bias in Meta-analysis: Prevention, Assessment and Adjustments. Sussex: JohnWiley & Sons, 2006

    2006

  23. [31]

    Scharzer, J

    G. Scharzer, J. Carpenter, and G. Rucker. Empirical evaluation suggests copas selection model preferable to trim-and-fill method for selection bias in meta-analysis. Journal of Clinical Epidemiology, 63: 0 282--288, 2010

    2010

  24. [32]

    J. A. Sterne, D. Gavaghan, and M. Egger. Publication and related bias in meta-analysis: power of statistical tests and prevalence in the literature. Journal Clinical Epidemiology, 53: 0 1119--1129, 2000

    2000

  25. [33]

    J .A. C. Sterne, M. Egger, and G. D. Smith. Systematic reviews in health care: Investigating and dealing with publication and other biases in meta-analysis. British Medical Journal, 323: 0 101--105, 2001

    2001

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.