A note on numerical evaluation of conditional Akaike information for nonlinear mixed-effects models

James T. Thorson; Nan Zheng; Noel Cadigan

REVIEW 2 major objections 4 minor 16 references

A note on numerical evaluation of conditional Akaike information for nonlinear mixed-effects models

T0 review · 2 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper derives two conditional-AIC estimators for nonlinear mixed-effects models and recommends Eq. (6) as the distributionally reliable one for any observation type.

desk verdict Practical cAIC estimator for nonlinear mixed models that works in simulations; the dropped O(1/nt) term deserves a real justification before the paper is accepted. read the letter →

arxiv 2411.14185 v1 pith:3SKYETBP submitted 2024-11-21 stat.ME math.STstat.TH

Nan Zheng , Noel Cadigan , James T. Thorson This is my paper · ORCID

classification stat.MEmath.STstat.TH

keywords conditionalAkaikeinformationnonlinearmixed-effectsmodelsmodelselectionbiascorrectionempiricalBayesTweediedistributionrandomeffectsautomaticdifferentiation

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to give practitioners a usable way to compute the conditional Akaike information criterion for nonlinear mixed-effects models, where cluster sizes are often small and observations may be skewed, discrete, or zero-inflated. It proposes two formulas: Method 1, a derivative-based correction for continuous data, and Method 2, a correction that assumes only that the modeled random-effect distribution is multivariate normal and is claimed to work for any observation type. The paper's central recommendation is Method 2, Eq. (6), whose penalty self-adjusts between counting only fixed parameters and counting fixed plus random effects. If the claim is right, one formula covers Gaussian, gamma, negative binomial, and Tweedie data with flexible links and small clusters, removing a major obstacle to model selection in nonlinear mixed models.

What carries the argument

The load-bearing identity is Theorem 2's Eq. (6), which expresses the optimism of the fitted conditional log-likelihood as $2p_c + 2q - 2\operatorname{trace}\{\ddot{\ell}_j^{-1}\ddot{\ell}_r\}$. Here $\ddot{\ell}_j$ is the second derivative of the joint log-likelihood with respect to the random effects and $\ddot{\ell}_r$ is the second derivative of the random-effect log-density, so the trace term is the effective number of random-effect parameters. When the data are uninformative about the random effects this trace is close to $q$ and the penalty shrinks toward $2p_c$; when the data pin the random effects down it is close to $0$ and the penalty approaches $2(p_c+q)$. The derivation reaches this trace by a second-order Taylor expansion of the conditional log-likelihood around the true parameters followed by an integration-by-parts identity for the multivariate normal random-effect model.

What would settle it

Use the paper's own Monte Carlo protocol (outer draws of true random effects, inner draws of prediction data) with tiny cluster sizes such as $n_t=2$, a skewed observation distribution, and a random-effect variance on the boundary of its parameter space. If the relative bias of Eq. (6)'s bias correction exceeds the paper's typical benchmark of roughly 0.035, or if adding the explicit $O(1/n_t)$ term from the Supplementary Material changes which model wins, the central recommendation fails.

Watch

Extended reading notes

Core claim

Working from the conditional Akaike information, the paper derives two estimators. Method 1 (Eq. 4) uses derivatives of the maximum marginal likelihood estimator $\hat{\theta}$ and the empirical Bayes predictor $\hat{\Psi}$ with respect to the data; the paper shows it is near-exact for Gaussian data with the identity link but degrades for skewed continuous data and cannot handle discrete or zero-inflated observations. Method 2 (Eq. 6) assumes the modeled random-effect distribution is multivariate normal and obtains the bias correction $2p_c + 2q - 2\operatorname{trace}\{\ddot{\ell}_j^{-1}\ddot{\ell}_r\} + O(1/n_t)$. In the paper's simulations this formula keeps relative bias small across normal, gamma, negative binomial, and Tweedie observations with log links and cluster sizes of 3 or 5, which is the stated basis for recommending Method 2.

Load-bearing premise

The claim stands on the assumption that the Taylor-series remainder and the $O(1/n_t)$ term dropped in Eq. (6) are small in practice; the paper supports this with an informal order argument in Appendix C and an unpublished numerical study, not with a proof or a published check.

Editorial extensions

If this is right

One formula, Eq. (6), applies to Gaussian, gamma, negative binomial, and Tweedie observations with flexible links and small bounded cluster sizes, so practitioners no longer need a bespoke cAIC derivation for each setting.
The criterion's effective penalty moves between $2p_c$ and $2(p_c+q)$ depending on how much information the data carry about the random effects, which lets it mimic both marginal and conditional AIC in the right limits.
The needed derivatives can be obtained numerically through automatic differentiation, so the formula is automatable for the kinds of nonlinear mixed models used in fisheries, ecology, pharmacology, and public health.
Method 1 remains a reasonable cAIC only for Gaussian data with identity links; the paper's simulations show it should not be trusted for skewed, discrete, or zero-inflated data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

As an editorial extension, including the explicit $O(1/n_t)$ expression derived in the Supplementary Material when cluster sizes are very small could be a sharp test of Eq. (6); the authors note their unpublished simulations saw no improvement, but the published evidence for that is absent.
Because Eq. (6) assumes only that the modeled random effects are multivariate normal, the same trace identity should transfer to latent Gaussian process and spline models, where that assumption is standard.
As an editorial extension, the derivative identities behind Method 1 are reusable for sensitivity analysis and delta-method standard errors, not only for cAIC.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Nan, quick take on arXiv:2411.14185. This is a genuinely useful note. The field has cAIC for linear and generalized linear mixed models, but nonlinear mixed-effects models with non-Gaussian, discrete, or zero-inflated observations lacked a workable criterion. The paper supplies two: Eq. (4) via Taylor expansion with derivatives of estimators w.r.t. data (Lemma 1), and Eq. (6), a simple trace correction that only needs the joint and RE-prior Hessians. Method 2 is the real contribution. It extends Yu–Yau and Liang et al. to NMMs with arbitrary links and small clusters, and the simulation evidence is consistent: relative biases in the bias-correction are small across Gaussian, gamma, negative binomial, and Tweedie cases, with cluster sizes 3 and 5. That is not a trivial result.

What I like: the paper is honest about what is approximate. It explicitly flags the O(1/nt) term in Eq. (6) and says why it might be small. It also ships working R/TMB code for the Gaussian example, which is a real artifact. The simulations are structured sensibly, and the 'true' BC is computed by simulation from the generating model, which is standard in this literature.

Now the soft spots, proportionate to how soft they are. The load-bearing concern is real: the drop of the O(1/nt) term and of the second-order differences in Appendix C is justified by an 'unpublished numerical study' and an informal order argument. For the small-cluster regime the paper itself emphasizes, that's not a proof. The paper would be stronger with either a bound on the term or the unpublished simulations included as a supplement. That said, the published simulations do empirically support the claim across many configurations, so the concern is not that the method fails; it's that the main theoretical justification for the drop is not in the paper. Second, the RB tables have no Monte Carlo standard errors, so we don't know the noise in those numbers. Minor. Third, code is only for the Gaussian case; the non-Gaussian and Tweedie simulations are not reproducible from the supplement. Also minor, but worth asking for.

The citation pattern looks fine. The Zheng–Cadigan self-citations are to published papers and are used for intermediate steps; that's appropriate.

Bottom line: this is a useful note, worth a serious referee. The recommendation is to accept after revision, with the authors asked to supply the unpublished simulation results or a theoretical argument for the dropped terms, plus code for at least one non-Gaussian case.

Referee Report

2 major / 4 minor

Summary. The manuscript proposes two estimators for the conditional Akaike information criterion (cAIC) in nonlinear mixed-effects models. Method 1 (Eq. 4) is a Taylor-expansion-based bias correction for continuous data and gives explicit derivative formulas for fixed-effects estimators and empirical Bayes random-effect predictors with respect to the data (Lemma 1). Method 2 (Eq. 6) assumes the random effects are modeled as multivariate normal and uses a Hessian-based trace correction. The paper reports simulation studies for Gaussian, gamma, negative binomial, and Tweedie observations with small fixed cluster sizes (nt = 3 or 5), evaluating the relative bias of the bias-correction term. Based on these simulations, the authors recommend Method 2 as a distributionally robust cAIC criterion for model selection.

Significance. If the central claim holds, the paper fills a practical gap by offering a cAIC formula applicable to nonlinear mixed-effects models with non-Gaussian, skewed, discrete, or zero-inflated responses, going beyond existing cAIC work that often assumes Gaussian or known-dispersion settings. The paper provides analytical derivative expressions, leverages TMB for automatic differentiation, and ships R/TMB code for the Gaussian example, which are concrete strengths. However, the central formulas rest on two approximations that are justified only by references to unpublished numerical studies, and the paper's own discussion acknowledges that the O(1/nt) term is not negligible in the small-cluster regime it emphasizes. These issues make the theoretical claim less secure than the simulations alone suggest.

major comments (2)

[§2.2, Eq. (6), and p.5] The O(1/nt) term in Eq. (6) is dropped on the basis of informal reasoning, but the paper's stated regime is small, bounded cluster sizes (nt = 3 or 5). The paper itself states that this term 'approaches zero only as cluster sizes increase' (p.5), and the subsequent two arguments—that differences between models are dominated by O(T) terms and that ∂Ψhat/∂θ^T is 'typically small'—do not constitute a bound. The only direct support is the assertion of 'unpublished simulation results' in the Conclusion (p.9). Since a non-negligible O(1/nt) term would make the bias correction in Eq. (6) itself biased, this is load-bearing for the headline recommendation of Method 2. The authors should either provide a rigorous condition under which the term is negligible, or present a reproducible numerical study that quantifies its magnitude across a grid of nt values and model configurations, or show that the simulation conclusions are insensitive to its inclusion.
[Appendix B (p.11) and Appendix C (p.13)] Both derivations neglect the difference between second-order Taylor terms. In Appendix B the text says, 'If we neglect the difference between the second order terms of (12) and (16) (our unpublished numerical study also indicates that the difference is small),' and in Appendix C the same step is introduced with 'Here we neglect the difference between the second order terms.' These are unverifiable approximations at the core of both Eq. (4) and Eq. (6). Because the paper is a methodological contribution, the justification cannot be an unpublished numerical study. Please include the numerical study in the supplementary material with sufficient detail to reproduce it, or provide a formal argument that the neglected difference is of lower order than the retained terms.

minor comments (4)

[§3, Tables 1-4] The relative-bias estimates are reported without Monte Carlo standard errors or confidence intervals. Given that many values are close to zero (e.g., -0.002 in Table 1), it is unclear how much of the deviation is Monte Carlo noise; adding standard errors would substantially improve interpretability.
[§3.1 and §3.2] The simulations for Eq. (4) and Eq. (6) use different numbers of outer and inner iterations (e.g., nout=1500/ninner=20000 for Eq. (6) versus nout=500/ninner=1000 for Eq. (4) in the Gaussian example). If the 'true BC' is recomputed with these different Monte Carlo sizes for each method, the comparison of the two methods mixes formula error with Monte Carlo error. Please state whether the same true-BC estimate was used for both methods and, if not, report the Monte Carlo uncertainty.
[Appendix C, p.12] There is a grammatical error: 'According to the the conditional inference' should read 'According to the conditional inference.'
[§2.2, p.5] The discussion of the O(1/nt) term first gives an expression involving the true random-effect covariance Σ_u and then notes that the MVN assumption for p(u) is unnecessary and that Σ_u^{-1} can be replaced by the model precision matrix; stating the final usable expression once in equation form would avoid ambiguity about which covariance is intended.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the cAIC estimators are analytic and benchmarked against Monte Carlo truth; self-citations are to independent published results, and the main weakness is an unverified smallness assumption, not circularity.

full rationale

The paper's central claims are the analytical cAIC estimators in Eqs. (4) and (6). These are derived from Taylor expansions of the conditional Akaike information under the stated assumption g(y|u) = f(y|θo, Ψo), with the necessary derivatives obtained by implicit differentiation in Lemma 1 and Appendix A. The derivations do not fit any parameter to the simulation outputs; the formulas are constructed before the simulations are used to evaluate them. The simulation study computes the 'true' bias correction by Monte Carlo integration over y* from the same generative model, which is the standard way to benchmark an estimator and does not make the estimator circular. The paper cites Zheng and Cadigan (2021, 2023) for intermediate expansions such as E(θ̂ − θo) ≈ 0 and conditional variance results. These are published peer-reviewed results by overlapping authors, but they are not restatements of the cAIC formula being derived; the cAIC claim does not reduce to these citations. The paper also invokes an 'unpublished numerical study' and 'unpublished simulation results' to justify neglecting second-order Taylor differences and the O(1/nt) term. This is an unsupported, load-bearing smallness assumption, especially for the small cluster sizes nt = 3 and 5 emphasized in the simulations, and it is a correctness risk rather than a circularity: dropping a term is an approximation, not an equivalence between the estimator and its own input. No equation in the paper is defined in terms of the quantity it purports to predict, and no fitted parameter is renamed as a prediction. The self-citations are present but do not constitute a circular chain, so the paper receives a low score reflecting minor self-citation without load-bearing circularity.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The method rests on standard cAIC assumptions plus a hand-waved smallness of neglected terms. No free parameters are introduced, and no new entities are postulated. The main burden is the unproven negligibility of higher-order terms, which is central to the validity of Eq. (6).

assumptions (4)

domain assumption The true data distribution g(y|u) equals the fitted model f(y|theta_o, Psi_o) for some theta_o and Psi_o.
Stated in Section 2.1; standard in cAIC derivations, but assumes no model misspecification.
domain assumption Random effects are modeled with a multivariate normal distribution in Method 2.
Required for Theorem 2 (Section 2.2); common in mixed-effects modeling.
ad hoc to paper The neglected second-order Taylor terms and the O(1/nt) term in Eq. (6) are small enough to ignore.
Appendix B and C invoke an 'unpublished numerical study' to justify neglecting terms; this is not proven in the paper.
standard math Well-defined second-order derivatives of the marginal and joint likelihoods exist.
Needed for Lemma 1; fails for discrete or zero-inflated models, which is why Method 2 is preferred there.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A note on numerical evaluation of conditional Akaike information for nonlinear mixed-effects models." pith.science (2026). https://pith.science/paper/3SKYETBP

@misc{pith2026241114185,
  author       = {Pith},
  title        = {Pith review of: A note on numerical evaluation of conditional Akaike information for nonlinear mixed-effects models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3SKYETBP}},
  note         = {Machine review of arXiv:2411.14185}
}

read the original abstract

We propose two methods to evaluate the conditional Akaike information (cAI) for nonlinear mixed-effects models with no restriction on cluster size. Method 1 is designed for continuous data and includes formulae for the derivatives of fixed and random effects estimators with respect to observations. Method 2, compatible with any type of observation, requires modeling the marginal (or prior) distribution of random effects as a multivariate normal distribution. Simulations show that Method 1 performs well with Gaussian data but struggles with skewed continuous distributions, whereas Method 2 consistently performs well across various distributions, including normal, gamma, negative binomial, and Tweedie, with flexible link functions. Based on our findings, we recommend Method 2 as a distributionally robust cAI criterion for model selection in nonlinear mixed-effects models.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 15 canonical work pages

[1]

Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In B. Petrov and F. Csaki (Eds.), Second International Symposium on Information Theory , pp.\ 267–81. Budapest: Akademiai Kiado

work page 1973
[2]

Cadigan, N., M. D. Robertson, K. Nirmalkanna, and N. Zheng (2022). The complex relationship between weight and length of atlantic cod off the south coast of newfoundland. Canadian Journal of Fisheries and Aquatic Sciences\/ 79\/ (11), 1798--1819

work page 2022
[3]

Cavanaugh, J. E. and A. A. Neath (2019). The akaike information criterion: Background, derivation, properties, application, interpretation, and refinements. Wiley Interdisciplinary Reviews: Computational Statistics\/ 11\/ (3), e1460

work page 2019
[4]

Hooten, M. B. and N. T. Hobbs (2015). A guide to bayesian model selection for ecologists. Ecological monographs\/ 85\/ (1), 3--28

work page 2015
[5]

Ingersoll, J. E. (1987). Theory of financial decision making , Volume 3. Rowman & Littlefield

work page 1987
[6]

Jorgensen, B. (1997). The theory of dispersion models . Chapman and Hall, London

work page 1997
[7]

Kass, R. E. and D. Steffey (1989). Approximate bayesian inference in conditionally independent hierarchical models (parametric empirical bayes models). Journal of the American Statistical Association\/ 84\/ (407), 717--726

work page 1989
[8]

Nielsen, C

Kristensen, K., A. Nielsen, C. W. Berg, H. Skaug, and B. Bell (2016). Tmb: automatic differentiation and laplace approximation. Journal of Statistical Software\/ 70\/ (5), 1–21

work page 2016

Show all 16 references

[9]

Wu, and G

Liang, H., H. Wu, and G. Zou (2008). A note on conditional aic for linear mixed-effects models. Biometrika\/ 95\/ (3), 773--778

2008
[10]

Lindstrom, M. J. and D. M. Bates (1990). Nonlinear mixed effects models for repeated measures data. Biometrics\/ , 673--687

1990
[11]

R: A Language and Environment for Statistical Computing

R Core Team (2022). R: A Language and Environment for Statistical Computing . Vienna, Austria: R Foundation for Statistical Computing

2022
[12]

Vaida, F. and S. Blanchard (2005). Conditional akaike information for mixed effects models. Biometrika\/ 92\/ (2), 351--370

2005
[13]

Wood, S. N., N. Pya, and B. S \"a fken (2016). Smoothing parameter and model selection for general smooth models. Journal of the American Statistical Association\/ 111\/ (516), 1548--1563

2016
[14]

Yu, D. and K. K. Yau (2012). Conditional akaike information criterion for generalized linear mixed models. Computational Statistics & Data Analysis\/ 56\/ (3), 629--644

2012
[15]

Zheng, N. and N. Cadigan (2021). Frequentist delta-variance approximations with mixed-effects models and tmb. Computational Statistics & Data Analysis\/ 160 , 107227

2021
[16]

Zheng, N. and N. Cadigan (2023). Frequentist conditional variance for nonlinear mixed-effects models. Journal of Statistical Theory and Practice\/ 17\/ (1), 1--30

2023

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In B. Petrov and F. Csaki (Eds.), Second International Symposium on Information Theory , pp.\ 267–81. Budapest: Akademiai Kiado

work page 1973

[2] [2]

Cadigan, N., M. D. Robertson, K. Nirmalkanna, and N. Zheng (2022). The complex relationship between weight and length of atlantic cod off the south coast of newfoundland. Canadian Journal of Fisheries and Aquatic Sciences\/ 79\/ (11), 1798--1819

work page 2022

[3] [3]

Cavanaugh, J. E. and A. A. Neath (2019). The akaike information criterion: Background, derivation, properties, application, interpretation, and refinements. Wiley Interdisciplinary Reviews: Computational Statistics\/ 11\/ (3), e1460

work page 2019

[4] [4]

Hooten, M. B. and N. T. Hobbs (2015). A guide to bayesian model selection for ecologists. Ecological monographs\/ 85\/ (1), 3--28

work page 2015

[5] [5]

Ingersoll, J. E. (1987). Theory of financial decision making , Volume 3. Rowman & Littlefield

work page 1987

[6] [6]

Jorgensen, B. (1997). The theory of dispersion models . Chapman and Hall, London

work page 1997

[7] [7]

Kass, R. E. and D. Steffey (1989). Approximate bayesian inference in conditionally independent hierarchical models (parametric empirical bayes models). Journal of the American Statistical Association\/ 84\/ (407), 717--726

work page 1989

[8] [8]

Nielsen, C

Kristensen, K., A. Nielsen, C. W. Berg, H. Skaug, and B. Bell (2016). Tmb: automatic differentiation and laplace approximation. Journal of Statistical Software\/ 70\/ (5), 1–21

work page 2016