Reliable model selection in the presence of parameter non-identifiability
Pith reviewed 2026-05-20 02:17 UTC · model grok-4.3
pith:NV7NVSC5 Add to your LaTeX paper
What is a Pith Number?\usepackage{pith}
\pithnumber{NV7NVSC5}
Prints a linked pith:NV7NVSC5 badge after your title and writes the identifier into PDF metadata. Compiles on arXiv with no extra files. Learn more
The pith
Adaptive multiple importance sampling provides reliable evidence estimates for model selection even with non-identifiable parameters.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
In the presence of parameter non-identifiability, deterministic evidence approximations violate their assumptions and yield inaccurate model selection. A novel implementation of adaptive multiple importance sampling for evidence estimation demonstrates robustness to non-identifiability. When applied to ecological case studies, the method produces model selection results comparable to Markov chain Monte Carlo methods while incurring substantially lower computational costs. Given the common occurrence of non-identifiability in mathematical biology, this offers a practical solution for reliable model selection.
What carries the argument
The adaptive multiple importance sampling procedure for evidence estimation, which dynamically adjusts proposal distributions to integrate over challenging posterior landscapes induced by non-identifiability.
Load-bearing premise
The adaptive multiple importance sampling procedure adequately explores and integrates over the posterior even when non-identifiability produces flat or multimodal surfaces without requiring problem-specific tuning.
What would settle it
A direct comparison on a known non-identifiable ecological model where the method's model rankings or evidence values diverge from those of extensive MCMC runs would falsify the robustness claim.
Figures
read the original abstract
Mathematical models are invaluable for understanding and predicting how biological systems behave, although their construction requires specifying mechanisms and relationships that are often not perfectly known. In the presence of multiple competing models, model uncertainty should be accounted for when performing inference based on available data. Bayesian model selection is a framework for testing mechanistic hypotheses and generating predictions under model uncertainty, which generally requires computation of the model evidence. In this work, we investigate the reliability of evidence computation methods when parameter non-identifiability -- the inability to distinguish between parameter values given available data -- is present, and find that deterministic evidence approximations can produce misleading model selection results because their underlying assumptions are violated. We propose a novel implementation of adaptive multiple importance sampling for evidence estimation, and demonstrate its robustness against non-identifiability. We use ecological case studies to demonstrate how simple model selection methods fail to produce accurate results, whereas our method yields model selection results that are comparable to those obtained by Markov chain Monte Carlo methods at substantially lower computational cost. Given the pervasiveness of parameter non-identifiability in mathematical biology, this work provides a practical approach to reliable model selection in the presence of poorly identified parameters.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript claims that deterministic evidence approximations for Bayesian model selection can produce misleading results in the presence of parameter non-identifiability. It proposes a novel implementation of adaptive multiple importance sampling (MIS) for evidence estimation that is robust to such non-identifiability, and uses ecological case studies to show that the method yields model selection results comparable to MCMC while incurring substantially lower computational cost.
Significance. If the adaptive MIS procedure reliably controls variance and produces consistent evidence estimates even on flat or multimodal posteriors without problem-specific tuning, the work would provide a practical, lower-cost alternative to MCMC for model selection in mathematical biology, where non-identifiability is common. The explicit comparison to established MCMC methods is a positive feature if the robustness claim is substantiated.
major comments (1)
- [Methods (adaptive MIS implementation)] Methods (adaptive MIS implementation): the adaptation of proposal distributions is not described in a manner that explicitly addresses flat or ridge-like posterior regions arising from non-identifiability. If the mechanism relies on empirical covariance or moment matching without regularization for unidentified directions, the importance weights can exhibit uncontrolled variance, which would invalidate the consistency of the evidence estimator and undermine the central robustness claim. This issue is load-bearing for the paper's main assertion.
minor comments (2)
- [Abstract] Abstract: the claim of robustness is supported only by case studies and MCMC comparison, yet no error analysis, convergence diagnostics, or full methodological detail is referenced, leaving the verification of the central claim incomplete from the given text.
- [Case studies] Case studies section: additional quantitative diagnostics (e.g., effective sample size or variance estimates of the evidence) would help readers evaluate whether the adaptive MIS truly explores the unidentified directions adequately.
Simulated Author's Rebuttal
We thank the referee for their detailed and constructive report. The major comment raises an important point about the clarity of our adaptive MIS description with respect to non-identifiability. We address it directly below and will incorporate the requested clarifications in a revised manuscript.
read point-by-point responses
-
Referee: Methods (adaptive MIS implementation): the adaptation of proposal distributions is not described in a manner that explicitly addresses flat or ridge-like posterior regions arising from non-identifiability. If the mechanism relies on empirical covariance or moment matching without regularization for unidentified directions, the importance weights can exhibit uncontrolled variance, which would invalidate the consistency of the evidence estimator and undermine the central robustness claim. This issue is load-bearing for the paper's main assertion.
Authors: We agree that the current manuscript description of the adaptation step would benefit from greater explicitness on this point. The adaptive procedure maintains a mixture of proposal distributions whose parameters are updated iteratively using weighted empirical moments from accumulated samples. To prevent degeneracy along unidentified directions, each covariance update includes a fixed small regularization term (a multiple of the identity matrix) chosen to ensure positive-definiteness while preserving the dominant directions of posterior mass. This construction is intended to keep the importance weights from exhibiting uncontrolled growth, consistent with the variance analysis provided in Section 3. We will revise the Methods section to add a dedicated paragraph (and accompanying pseudocode) that explicitly describes the regularization step, its motivation from non-identifiability considerations, and the resulting bound on the second moment of the weights. We will also include a brief numerical illustration on a simple ridge-like posterior to demonstrate controlled variance. revision: yes
Circularity Check
No circularity: novel adaptive MIS method validated empirically against external MCMC benchmarks
full rationale
The paper proposes a new adaptive multiple importance sampling procedure for evidence estimation and tests its robustness on ecological models with non-identifiability. Validation relies on direct numerical comparisons to standard MCMC (an externally established method) rather than any self-referential definitions, fitted inputs renamed as predictions, or load-bearing self-citations. No equations or claims reduce by construction to the authors' prior outputs; the central performance claims are supported by case-study results that remain falsifiable outside the paper's own fitted values.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Forecasting pathogen dynamics with
Abboud, Candy and Parent, Eric and Bonnefon, Olivier and Soubeyrand, Samuel , year = 2023, month = jun, journal =. Forecasting pathogen dynamics with. doi:10.1007/s11538-023-01169-w , urldate =
-
[2]
Computer Physics Communications , volume =
A computational approach to the structural analysis of uncertain kinetic systems , author =. Computer Physics Communications , volume =. doi:10.1016/j.cpc.2018.03.002 , urldate =
-
[3]
Model selection for ecologists: the worldviews of
Aho, Ken and Derryberry, DeWayne and Peterson, Teri , year = 2014, journal =. Model selection for ecologists: the worldviews of. doi:10.1890/13-1452.1 , urldate =
-
[4]
IEEE Transactions on Automatic Control , volume =
A new look at the statistical model identification , author =. IEEE Transactions on Automatic Control , volume =. doi:10.1109/TAC.1974.1100705 , urldate =
-
[5]
Statistics and Computing , volume =
Convergence rates for optimised adaptive importance samplers , author =. Statistics and Computing , volume =. doi:10.1007/s11222-020-09983-1 , urldate =
-
[6]
Amador, Andreia and Fernandes, Florbela P. and Santos, Lino O. and Romanenko, Andrey and Rocha, Ana Maria A. C. , editor =. Parameter estimation of the kinetic \ \ alpha\ \ -pinene isomerization model using the. Computational. doi:10.1007/978-3-319-95165-2_44 , abstract =
-
[7]
Arceo, Carlene Perpetua P. and Jose, Editha C. and. Chemical reaction network approaches to. Mathematical Biosciences , volume =. doi:10.1016/j.mbs.2015.08.022 , urldate =
-
[8]
Sparse and smooth signal estimation:
Atamturk, Alper and Gomez, Andres and Han, Shaoning , year = 2021, journal =. Sparse and smooth signal estimation:
work page 2021
-
[9]
Proceedings of the National Academy of Sciences , volume =
Topological sensitivity analysis for systems biology , author =. Proceedings of the National Academy of Sciences , volume =. doi:10.1073/pnas.1414026112 , urldate =
-
[10]
Banga, Julio R. and Villaverde, Alejandro F. , year = 2025, month = dec, journal =. Mechanistic dynamic modelling of biological systems:. doi:10.1016/j.coisb.2025.100553 , urldate =
-
[11]
PLOS Computational Biology , volume =
Unified tumor growth mechanisms from multimodel inference and dataset integration , author =. PLOS Computational Biology , volume =. doi:10.1371/journal.pcbi.1011215 , urldate =
-
[12]
and Jasra, Ajay and Whiteley, Nick , year = 2014, month = mar, journal =
Beskos, Alexandros and Crisan, Dan O. and Jasra, Ajay and Whiteley, Nick , year = 2014, month = mar, journal =. Error bounds and normalising constants for sequential. doi:10.1239/aap/1396360114 , urldate =
-
[13]
The horseshoe-like regularization for feature subset selection , author =. Sankhya B , volume =. doi:10.1007/s13571-019-00217-7 , urldate =
-
[14]
Bhatt, Nirav and Jayawardhana, Bayu and Plaza, Santiago S. 2023 62nd. doi:10.1109/CDC49753.2023.10384032 , urldate =
-
[15]
Bishop, Christopher M. , year = 2006, publisher =. Mixture. Pattern
work page 2006
-
[16]
Computers & Chemical Engineering , volume =
A global optimization method for model selection in chemical reactions networks , author =. Computers & Chemical Engineering , volume =. doi:10.1016/j.compchemeng.2016.05.016 , urldate =
-
[17]
Bock, H. G. , editor =. Numerical treatment of inverse problems in chemical reaction kinetics , booktitle =. doi:10.1007/978-3-642-68220-9_8 , abstract =
-
[18]
Some problems associated with the analysis of multiresponse data , author =. Technometrics , volume =. doi:10.2307/1266823 , urldate =. 1266823 , eprinttype =
-
[19]
PLOS Computational Biology , volume =
Geometric analysis enables biological insight from complex non-identifiable models using simple surrogates , author =. PLOS Computational Biology , volume =. doi:10.1371/journal.pcbi.1010844 , urldate =
-
[20]
Journal of The Royal Society Interface , volume =
Identifiability analysis for stochastic differential equation models in systems biology , author =. Journal of The Royal Society Interface , volume =. doi:10.1098/rsif.2020.0652 , urldate =
-
[21]
Electronic Journal of Statistics , volume =
A tracking approach to parameter estimation in linear ordinary differential equations , author =. Electronic Journal of Statistics , volume =. doi:10.1214/15-EJS1086 , urldate =
-
[22]
Proceedings of the National Academy of Sciences of the United States of America , volume =
Discovering governing equations from data by sparse identification of nonlinear dynamical systems , author =. Proceedings of the National Academy of Sciences of the United States of America , volume =. doi:10.1073/pnas.1517384113 , abstract =
-
[23]
, year = 2021, month = sep, journal =
Buchholz, Alexander and Chopin, Nicolas and Jacob, Pierre E. , year = 2021, month = sep, journal =. Adaptive tuning of. doi:10.1214/20-BA1222 , urldate =
-
[24]
Nested sampling methods , author =. Statistics Surveys , volume =. doi:10.1214/23-SS144 , urldate =
-
[25]
IEEE Signal Processing Magazine , author =
Bugallo, Monica F. and Elvira, Victor and Martino, Luca and Luengo, David and Miguez, Joaquin and Djuric, Petar M. , year = 2017, month = jul, journal =. Adaptive importance sampling:. doi:10.1109/MSP.2017.2699226 , urldate =
-
[26]
Burda, Martin and Daviet, Remi , year = 2023, month = jan, journal =. Hamiltonian sequential. doi:10.1080/07474938.2022.2140982 , urldate =
-
[27]
Burnham, Kenneth P. and Anderson, David R. and Huyvaert, Kathryn P. , year = 2011, month = jan, journal =. doi:10.1007/s00265-010-1029-6 , urldate =
-
[28]
Chemical Engineering Science , volume =
Inference of chemical reaction networks , author =. Chemical Engineering Science , volume =. doi:10.1016/j.ces.2007.10.010 , urldate =
-
[29]
Burrage, Pamela M. and Weerasinghe, Hasitha N. and Burrage, Kevin , year = 2024, month = jul, journal =. Using a library of chemical reactions to fit systems of ordinary differential equations to agent-based models: a machine learning approach , shorttitle =. doi:10.1007/s11075-023-01737-0 , urldate =
-
[30]
The horseshoe estimator for sparse signals , author =. Biometrika , volume =. doi:10.1093/biomet/asq017 , urldate =. 25734098 , eprinttype =
-
[31]
Bulletin of Mathematical Biology , volume =
A continuation technique for maximum likelihood estimators in biological models , author =. Bulletin of Mathematical Biology , volume =. doi:10.1007/s11538-023-01200-0 , urldate =
-
[32]
Cedersund, Gunnar and Roll, Jacob , year = 2009, journal =. Systems biology: model based evaluation and comparison of potential explanations for given biological data , shorttitle =. doi:10.1111/j.1742-4658.2008.06845.x , urldate =
-
[33]
A unified sparse optimization framework to learn parsimonious physics-informed models from data , author =. IEEE Access , volume =. doi:10.1109/ACCESS.2020.3023625 , abstract =
-
[34]
Champneys, Max D. and Rogers, Timothy J. , year = 2025, month = feb, eprint =. doi:10.1098/rspa.2024.0620 , urldate =
-
[35]
Journal of Machine Learning Research , volume =
Optimization over a probability simplex , author =. Journal of Machine Learning Research , volume =
-
[36]
The. Model. doi:10.1017/CBO9780511790485.004 , urldate =
-
[37]
Scandinavian Journal of Statistics , volume =
Adaptive multiple importance sampling , author =. Scandinavian Journal of Statistics , volume =. doi:10.1111/j.1467-9469.2011.00756.x , urldate =
-
[38]
SIAM/ASA Journal on Uncertainty Quantification , volume =
Ensemble transport adaptive importance sampling , author =. SIAM/ASA Journal on Uncertainty Quantification , volume =. doi:10.1137/17M1114867 , urldate =
-
[39]
SIAM/ASA Journal on Uncertainty Quantification , volume =
Transport map accelerated adaptive importance sampling, and application to inverse problems arising from multiscale stochastic reaction networks , author =. SIAM/ASA Journal on Uncertainty Quantification , volume =. doi:10.1137/19M1239416 , urldate =
-
[40]
Journal of Mathematical Chemistry , volume =
Identifiability of chemical reaction networks , author =. Journal of Mathematical Chemistry , volume =. doi:10.1007/s10910-007-9307-x , urldate =
-
[41]
Mathematical Biosciences and Engineering , volume =
Realizations of kinetic differential equations , author =. Mathematical Biosciences and Engineering , volume =. doi:10.3934/mbe.2020046 , urldate =
-
[42]
Journal of The Royal Society Interface , volume =
Understanding the impact of numerical solvers on inference for differential equation models , author =. Journal of The Royal Society Interface , volume =. doi:10.1098/rsif.2023.0369 , urldate =
-
[43]
and Whiteley, Nick , year = 2022, month = sep, journal =
Dai, Chenguang and Heng, Jeremy and Jacob, Pierre E. and Whiteley, Nick , year = 2022, month = sep, journal =. An invitation to sequential. doi:10.1080/01621459.2022.2087659 , urldate =
-
[44]
Das, Swagatam and Maity, Sayan and Qu, Bo-Yang and Suganthan, P. N. , year = 2011, month = jun, journal =. Real-parameter evolutionary multimodal optimization ---. doi:10.1016/j.swevo.2011.05.005 , urldate =
-
[45]
Del Moral, Pierre and Doucet, Arnaud and Jasra, Ajay , year = 2006, month = jun, journal =. Sequential. doi:10.1111/j.1467-9868.2006.00553.x , urldate =
-
[46]
Demasi, Gabriele and Capurri, Giulia and Lenti, Massimo and Ricciardone, Angelo and Patricelli, Barbara and Mascioli, Adriano Frattale and Piccari, Lorenzo and Albuquerque, Saulo and Guidi, Gianluca M. and Pannarale, Francesco and Stratta, Giulia and Pozzo, Walter Del , year = 2026, month = jan, number =. The. doi:10.48550/arXiv.2601.02336 , urldate =. ar...
-
[47]
and Maskell, Simon , year = 2024, journal =
Devlin, Lee and Carter, Matthew and Horridge, Paul and Green, Peter L. and Maskell, Simon , year = 2024, journal =. The no-. doi:10.1109/LSP.2024.3386494 , urldate =
-
[48]
Diciccio, Thomas J. and Kass, Robert E. and Raftery, Adrian and Wasserman, Larry , year = 1997, month = sep, journal =. Computing. doi:10.1080/01621459.1997.10474045 , urldate =
-
[49]
Ding, Jie and Tarokh, Vahid and Yang, Yuhong , year = 2018, month = nov, journal =. Model selection techniques:. doi:10.1109/MSP.2018.2867638 , urldate =
-
[50]
Notes on the practical application of nested sampling:
Dittmann, Alexander , year = 2024, month = sep, journal =. Notes on the practical application of nested sampling:. doi:10.33232/001c.123872 , urldate =
-
[51]
Dormann, Carsten F. and Calabrese, Justin M. and. Model averaging in ecology: a review of. Ecological Monographs , volume =. doi:10.1002/ecm.1309 , urldate =
-
[52]
Journal of the Royal Statistical Society
Assessment and propagation of model uncertainty , author =. Journal of the Royal Statistical Society. Series B (Methodological) , volume =. doi:10.1111/j.2517-6161.1995.tb02015.x , urldate =. 2346087 , eprinttype =
-
[53]
Communications Physics , volume =
Automatically discovering ordinary differential equations from data with sparse regression , author =. Communications Physics , volume =. doi:10.1038/s42005-023-01516-2 , urldate =
-
[54]
Industrial & Engineering Chemistry Research , volume =
Dynamic optimization of nonlinear processes with an enhanced scatter search method , author =. Industrial & Engineering Chemistry Research , volume =. doi:10.1021/ie801717t , urldate =
-
[55]
Elvira, V. Advances in. Wiley. doi:10.1002/9781118445112.stat08284 , urldate =
-
[56]
IEEE Signal Processing Letters , volume =
Efficient multiple importance sampling estimators , author =. IEEE Signal Processing Letters , volume =. doi:10.1109/LSP.2015.2432078 , urldate =
-
[57]
Statistical Science , author =
Generalized multiple importance sampling , author =. Statistical Science , volume =. doi:10.1214/18-STS668 , urldate =
-
[58]
Journal of Global Optimization , volume =
Deterministic global optimization in nonlinear optimal control problems , author =. Journal of Global Optimization , volume =. doi:10.1023/A:1026578104213 , urldate =
-
[59]
Fasel, U. and Kutz, J. N. and Brunton, B. W. and Brunton, S. L. , year = 2022, month = apr, journal =. Ensemble-. doi:10.1098/rspa.2021.0904 , urldate =
-
[60]
, year = 2021, month = sep, pages =
Frisch, Daniel and Hanebeck, Uwe D. , year = 2021, month = sep, pages =. Gaussian mixture estimation from weighted samples , booktitle =. doi:10.1109/MFI52462.2021.9591161 , urldate =
-
[61]
Estimating marginal likelihoods for mixture and
Fr. Estimating marginal likelihoods for mixture and. The Econometrics Journal , volume =. doi:10.1111/j.1368-423X.2004.00125.x , urldate =
-
[62]
Fuguitt, Robert E. and Hawkins, J. Erskine , year = 1945, month = feb, journal =. The liquid phase thermal isomerization of. doi:10.1021/ja01218a031 , urldate =
-
[63]
Fuguitt, Robert E. and Hawkins, J. Erskine , year = 1947, month = feb, journal =. Rate of the thermal isomerization of. doi:10.1021/ja01194a047 , urldate =
-
[64]
Chemical Engineering Science , volume =
Bayesian inference of chemical kinetic models from proposed reactions , author =. Chemical Engineering Science , volume =. doi:10.1016/j.ces.2014.10.030 , urldate =
-
[65]
Bayesian inference of chemical reaction networks , author =
-
[66]
Journal of The Royal Society Interface , volume =
Exploiting network topology for large-scale inference of nonlinear reaction models , author =. Journal of The Royal Society Interface , volume =. doi:10.1098/rsif.2018.0766 , urldate =
-
[67]
Gelfand, A. E. and Dey, D. K. , year = 1994, month = jan, journal =. Bayesian model choice: asymptotics and exact calculations , shorttitle =. doi:10.1111/j.2517-6161.1994.tb01996.x , urldate =
-
[68]
Gelman, Andrew and Carlin, John B. and Stern, Hal S. and Dunson, David B. and Vehtari, Aki and Rubin, Donald B. , year = 2013, month = nov, edition =. Bayesian. doi:10.1201/b16018 , abstract =
-
[69]
Gelman, Andrew and Carlin, John B. and Stern, Hal S. and Dunson, David B. and Vehtari, Aki and Rubin, Donald B. , year = 2013, edition =. Modal and distributional approximations , booktitle =
work page 2013
-
[70]
Simulating normalizing constants:
Gelman, Andrew and Meng, Xiao-Li , year = 1998, journal =. Simulating normalizing constants:. 2676756 , eprinttype =
work page 1998
-
[71]
Predictive power of non-identifiable models , author =. Scientific Reports , volume =. doi:10.1038/s41598-023-37939-8 , urldate =
-
[72]
Green, Peter J. , year = 1995, journal =. Reversible jump. doi:10.2307/2337340 , urldate =. 2337340 , eprinttype =
-
[73]
Variance-aware multiple importance sampling , author =. ACM Trans. Graph. , volume =. doi:10.1145/3355089.3356515 , urldate =
-
[74]
PLOS Computational Biology , volume =
Parallel tempering with lasso for model reduction in systems biology , author =. PLOS Computational Biology , volume =. doi:10.1371/journal.pcbi.1007669 , urldate =
-
[75]
PLOS Computational Biology , volume =
Universally sloppy parameter sensitivities in systems biology models , author =. PLOS Computational Biology , volume =. doi:10.1371/journal.pcbi.0030189 , urldate =
-
[76]
The Annals of Statistics , volume =
On the choice of a model to fit data from an exponential family , author =. The Annals of Statistics , volume =. doi:10.1214/aos/1176350709 , urldate =
-
[77]
On structural and practical identifiability:
Heinrich, Mio and Rosenblatt, Marcus and Wieland, Franz-Georg and Stigter, Hans and Timmer, Jens , year = 2025, month = jun, journal =. On structural and practical identifiability:. doi:10.1016/j.coisb.2025.100546 , urldate =
-
[78]
Hines, Keegan E. and Middendorf, Thomas R. and Aldrich, Richard W. , year = 2014, month = feb, journal =. Determination of parameter identifiability in nonlinear biophysical models:. doi:10.1085/jgp.201311116 , urldate =
-
[79]
Hirsh, Seth M. and. Sparsifying priors for. Royal Society Open Science , volume =. doi:10.1098/rsos.211823 , urldate =
-
[80]
and Madigan, David and Raftery, Adrian E
Hoeting, Jennifer A. and Madigan, David and Raftery, Adrian E. and Volinsky, Chris T. , year = 1999, month = nov, journal =. Bayesian model averaging: a tutorial (with comments by. doi:10.1214/ss/1009212519 , urldate =
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.