Pith. sign in

REVIEW 2 major objections 5 minor 38 references

Simulation-based validation of Bayes factor computation

T0 review · 2 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper proves that validating Bayes factor computation reduces to checking one binary prediction: whether the computed model probability is calibrated.

desk verdict SBC for the BMA supermodel is a real advance for Bayes factor validation, and the binary-calibration equivalence is close to right—but the paper states it more broadly than the theorem proves. read the letter →

arxiv 2508.11814 v4 pith:YINZZ3EU submitted 2025-08-15 stat.ME

Martin Modrák , Sebastian Stroppel , Paul-Christian Bürkner This is my paper · ORCID
classification stat.ME MSC 62C10
keywords Bayesfactorsimulation-basedcalibrationbinarypredictionBayesianmodelaveragingmarginallikelihoodimproperpriorsvalidation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper's central claim is that Bayes factor computation can be validated by treating the competing models as one Bayesian model-averaging supermodel and checking the calibration of the model index: when the computed posterior probability for a model is $p$, that model should be the true one in $p$ of simulated cases. It proves a theoretical equivalence: for binary test quantities, continuous simulation-based calibration (SBC) and binary prediction calibration are the same check, provided the distribution of predicted probabilities is not singular. Empirically, the paper shows that with finite simulations binary calibration is often the more sensitive of the two, while SBC with data-dependent test quantities catches problems such as ignored data, noisy Bayes factors, and biased normalization constants that both binary calibration and data-averaged posterior checks miss. It further shows that the Good check, as originally described, does not control error rates, and that a posterior version of SBC is needed when models have improper priors. For the standard scenarios tested, the paper finds that the Bayes factor implementations it examined satisfy all available checks.

What carries the argument

The machinery is the BMA supermodel: a hierarchical model in which the model index $i$ is drawn from a categorical prior, parameters are drawn from the chosen submodel's prior, and data are drawn from that submodel's likelihood. The posterior probability of the model index is a one-to-one transform of the Bayes factor via Bayes' theorem, so checking correctness of one checks the other. Two test statistics carry the argument: continuous SBC, which ranks the prior draw within the candidate posterior and demands uniformity, and binary prediction calibration, which demands $\Pr(i=k \mid D_k=p)=p$ for the reported probability $D_k$. Theorem 6 is the pivot: for binary test quantities these two conditions coincide whenever the distribution of $D_k$ has no singular part, so the practical choice between them is a matter of sensitivity and finite-sample power.

What would settle it

Simulate a deliberately broken Bayes factor computation in which the reported posterior model probabilities take only two values, such as 0.5 and 1, by e.g. ignoring half the data in a Poisson-versus-negative-binomial comparison, then run both checks; binary prediction calibration should report near-perfect calibration while the SBC gamma statistic on a data-dependent quantity rejects, confirming the non-singularity condition is doing real work.

Watch

Extended reading notes

Core claim

The discovery is a direct reduction: a correct Bayes factor computation is equivalent to calibrated predictions about which model generated the data. Working inside the BMA supermodel, the paper defines binary prediction calibration as the condition that, among all simulated datasets for which an algorithm reports posterior model probability $p$, the model it favors is indeed the data-generating model in exactly a proportion $p$ of them. Theorem 6 states that this condition holds if and only if the candidate posterior passes continuous SBC with respect to the binary test quantity $f(\cdot,y) = \text{model index}$, with the caveat that the distribution of predicted probabilities must not be singular. The paper also shows that passing calibration forces the correct data-averaged posterior, so data-averaged checking is a strictly weaker test in the infinite-simulation limit; conversely, SBC with suitably chosen test quantities can detect any mismatch, including ones invisible to binary calibration. On the practical side, the paper demonstrates that posterior SBC resolves apparent miscalibration caused by improper priors, and it validates widely used Bayes factor software on several real models.

Load-bearing premise

The equivalence between the two checks assumes the algorithm's predicted model probabilities have a distribution without singular atoms; if predictions concentrate on a few isolated values, binary calibration can pass while SBC with a data-dependent test quantity flags a problem.

Editorial extensions

If this is right

  • Bayes factor software can be validated end-to-end by checking one binary prediction, requiring only simulations from the prior and the models, with several hundred simulations as a practical minimum.
  • Data-averaged posterior checks should be used only as a supplement: they add no information in the infinite-simulation limit and miss flipped model indices, ignored data, and noisy computations.
  • The Good check as originally implemented should not be relied on for error control, because Bayes factor distributions can have infinite or very large variance.
  • When models contain improper priors, estimating a posterior SBC prior from a fixed dataset is necessary; using fixed or surrogate values for shared parameters produces false miscalibration.
  • For the standard scenarios tested, the Bayes factor packages examined pass all available checks, suggesting they are safe for routine low-stakes use.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the non-singularity caveat in Theorem 6 means practitioners should not choose between SBC and binary calibration solely by theoretical equivalence; running both and reporting disagreements is a cheap diagnostic for atom-heavy prediction distributions.
  • Editorial inference: the same binary-calibration logic could be applied directly to other reported quantities like posterior predictive $p$-values or confidence-interval coverage, since any binary event can play the role of the model index.
  • Editorial inference: the sensitivity contrast between data-dependent SBC and model-index checks suggests that future validation workflows should include at least one test quantity that is a function of the data itself, not just parameters.
  • Editorial inference: the slow convergence of the Good check implies moment-based validation is generally fragile for heavy-tailed Bayes factor distributions; rank-based or calibration-based checks are better suited to a validation standard.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper develops and evaluates simulation-based checks for Bayes factor computation. It introduces an improved variant of simulation-based calibration checking (SBC) applied to the Bayesian model averaging (BMA) supermodel, and proposes binary prediction calibration of the model index as a complementary check. The main theoretical result (Theorem 6 in Appendix A) establishes that, for binary test quantities, binary prediction calibration implies continuous SBC, and the converse holds under a non-singularity condition on the distribution of binary predictions. The authors also analyse data-averaged posterior checking and the Good check, showing limitations of the latter, and they introduce posterior SBC for settings with improper priors. Extensive simulation studies on toy models and realistic examples validate the bridgesampling and BayesFactor R packages. The paper includes reproducible code.

Significance. The paper provides a practical and theoretically grounded toolkit for validating Bayes factor computations, a topic of increasing importance. The empirical comparison is thorough, the code is publicly available, and the demonstration that the Good check lacks error control and that data-averaged posterior checking can be blind to important failure modes is valuable. The posterior SBC extension for improper priors is a useful contribution. The main theoretical equivalence result is elegant, but the narrative overstates its scope (see major comment below), so the central claim needs qualification before the paper can be accepted.

major comments (2)
  1. [Section 3.2 and Abstract] The paper states in Section 3.2 that for K=2, SBC for the model index in the BMA supermodel is "in fact the same as binary prediction calibration", and the abstract claims that "in theory, binary prediction calibration is equivalent to a special case of SBC". This is stronger than what is proved. Theorem 6 in Appendix A proves that binary prediction calibration implies continuous SBC unconditionally, but the converse requires that the distribution of D_{φ,f} is not singular. In the paper's own discrete-data examples (Section 5.1 with a binary observation, Section 5.2 with Poisson/NB counts), D takes finitely or countably many values, so the distribution is singular, and the converse is not covered by the theorem as stated. The unqualified equivalence is subsequently used in Section 7.1 to justify preferring binary prediction calibration over SBC for the model index. Please qualify the equivalence in the abstract and Section 3.2 by stating the non-singularity condition, and either prove the converse for the atomic case or explicitly note that the two checks may in principle disagree when D is discrete.
  2. [Section 3.2 and Abstract] The claim that "with well-designed test quantities, SBC can however detect all possible problems in computation" is a strong theoretical statement. It is credible given the arguments in Modrák et al. (2025), but the paper should make explicit that this is an in-principle statement requiring the user to choose suitable test quantities, and that finite simulation budgets and practical choice of test quantities will limit sensitivity. A brief clarification would prevent overinterpretation by practitioners.
minor comments (5)
  1. [Section 3.2 and Appendix A] The main text refers to "Theorem 1 in Appendix A" for the SBC–binary calibration equivalence, but the theorem is numbered Theorem 6 in the appendix. Similarly, "Theorem 3 in Appendix A" in Section 3.4 appears to refer to Theorem 9. Please fix these cross-references.
  2. [Figures 2 and 3] The captions state that the vertical orange line "marks when the check first attains 80% power", but the procedure for computing this point from the 100 resampled histories is not described. Please clarify how the 80% power point is defined and how the shared simulation pool is used.
  3. [Section 7.3] The recommendation that "at least several hundred simulations per validation scenario should be run" is reasonable, but given that some of the paper's own scenarios require 20,000 simulations to detect mild miscalibration (Appendix E), consider adding a more explicit warning that the required number depends on the severity and type of the potential problem.
  4. [Abstract and Section 7.3] There are several places where "bridgesamplingandBayesFactorR" appears without proper spacing, and the phrase "or a warning on inaccuracy" is unclear; please specify what kind of warning was observed.
  5. [Section 2.2] The sentence "Modrák et al. (2025) show that checking continuous SBC is equivalent to checking M-sample SBC for all M" could be misread as claiming equivalence for a single finite M; rephrase to clarify that passing continuous SBC is equivalent to passing M-sample SBC for every M simultaneously.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the equivalence theorem is proved in the appendix, and the empirical checks target independently implemented packages.

full rationale

The paper's central theoretical claim is Theorem 6 in Appendix A, which proves the relationship between binary prediction calibration and continuous SBC directly from Definitions 3-5 using explicit calculations; it is not imported as an assumption from prior work. Section 3.2's application to the BMA supermodel is a substitution into that theorem rather than a definitional restatement. The SBC foundations taken from Modrák et al. (2025) are background machinery, and although this is a self-citation, it is not used to force the paper's novel equivalence: the equivalence proof is self-contained in the manuscript. The empirical validation targets the external bridgesampling and BayesFactor R packages, and the toy examples construct discrepancies independently of any fitted parameter being renamed as a prediction. The unqualified equivalence stated in Section 3.2 omits the non-singularity condition required for the converse direction of Theorem 6, but that is an overprecision/correctness issue rather than a circular derivation. No step in the claimed validation reduces to its own inputs by construction, so no circularity is present.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper introduces no new entities or fitted parameters. It relies on standard probability theory, the BMA supermodel construction, and a non-singularity assumption on the distribution of predicted probabilities for the equivalence theorem.

assumptions (5)
  • domain assumption The candidate posterior family \u03d5 is a normalized probability density for every data point y (Definition 1).
    The entire validation theory treats the output of a Bayes factor computation as a normalized posterior over model index and parameters; if the computation outputs an improper distribution, the checks do not apply. Definition 1 in Appendix A.
  • standard math Exchangeability of prior and posterior draws under the correct model (Eq. 8).
    SBC relies on the fact that for a correct posterior, the prior draw that generated the data is exchangeable with posterior draws. This is a standard result of Bayesian consistency, stated in Section 2.2.
  • ad hoc to paper The distribution of binary predictions D\u03d5,f is non-singular for the equivalence between SBC and binary prediction calibration (Theorem 6).
    The proof of the second direction (SBC implies binary calibration) requires that the distribution of D has a density plus possibly a mass function, not a singular distribution. If predictions are degenerate (e.g., always 0.5), the equivalence may fail. Stated in Theorem 6 and Appendix A.
  • domain assumption The BMA supermodel is a valid probability model with well-defined marginal likelihoods.
    The entire approach checks the BMA supermodel, which requires each submodel's marginal likelihood to exist. This is standard in Bayesian model comparison, Section 2.1.
  • standard math All checks are interpreted in the limit of infinitely precise computation and infinitely many simulations.
    The theoretical guarantees are asymptotic; finite-sample behavior is evaluated empirically. Stated in Section 2.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Simulation-based validation of Bayes factor computation." pith.science (2026). https://pith.science/paper/YINZZ3EU

@misc{pith2026250811814,
  author       = {Pith},
  title        = {Pith review of: Simulation-based validation of Bayes factor computation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YINZZ3EU}},
  note         = {Machine review of arXiv:2508.11814}
}
read the original abstract

We propose and evaluate two methods that validate the computation of Bayes factors: one based on an improved variant of simulation-based calibration checking (SBC) and one based on calibration metrics for binary predictions. We show that in theory, binary prediction calibration is equivalent to a special case of SBC, but with limited resources, binary prediction calibration is typically more sensitive to the problems we investigated. With well-designed test quantities, SBC can however detect all possible problems in computation, including some that cannot be uncovered by binary prediction calibration. Previous work on Bayes factor validation includes checks based on the data-averaged posterior and the Good check method. We demonstrate that both checks miss many problems in Bayes factor computation detectable with SBC and binary prediction calibration. Moreover, we find that the Good check as originally described fails to control its error rates. Our proposed checks also typically use simulation results more efficiently than data-averaged posterior checks. Finally, we show that a special approach based on posterior SBC is necessary when checking Bayes factor computation under improper priors and we validate several models with such priors. We recommend that novel methods for Bayes factor computation be validated with SBC, binary prediction calibration and data-averaged posterior with at least several hundred simulations. For all the models we tested, the bridgesampling and BayesFactor R packages satisfy all available checks and thus are likely safe to use in standard scenarios.

Figures

Figures reproduced from arXiv: 2508.11814 by the authors.

Figure 1
Figure 1. Histories of check statistics for the correct binary model (A - C) and for the flipped binary model (D - F). The horizontal blue line marks rejecting perfect calibration with 5% false positive rate. For the correct model, we see that in all cases, violations tend to be non-severe and short-lived. For the flipped model SBC as well as miscalibration identify the problem very quickly, while data-averaged posterior chec… view at source ↗
Figure 2
Figure 2. Histories of SBC gamma statistic for poisson-NB model ignoring all data in BF computation (A) and ignoring half of the data (B). The statistic is reported for the model index and two data-dependent derived quantities: the log likelihood (log_lik) and variance estimate (var_y). We see that the model index fails to detect the problem, while log likelihood is relatively quick to detect it followed by the variance estim… view at source ↗
Figure 3
Figure 3. Histories of check statistics for poisson-NB model with noise in BF (A - C) and with bias in BF (D - F). A) and D) show the log-gamma statistic of default SBC, B) and E) is the p-value of the bootstrapped miscalibration test of Dimitriadis et al. (2021) and C) and F) is the p-value from t-test for data-averaged posterior checking (DAP). The horizontal blue line marks rejecting perfect calibration with 5% false posit… view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: Histories of check statistics for bad normalization constant in the turtles model. A) the log-gamma statistic of SBC, (lp__ = log-posterior density), B) the p-value of the bootstrapped miscalibration test and C) the p-value from t-test for data-averaged posterior check…
Figure 5
Figure 5. Figure 5: Histories of check statistics for random effect model using constant intercept and standard deviation. A) shows the log-gamma statistic of default SBC for the model index and two of the random intercepts (γ), B) is the p-value of the bootstrapped miscalibration test of…
Figure 6
Figure 6. Figure 6: Result of calibration checks for random effect model via the BayesFactor package, using posterior SBC. A) empirical CDF difference plots, B) binary calibration plot, horizontal axis is the predicted probability for model 1, vertical axis is the conditional event probab…
Figure 7
Figure 7. Figure 7: Results of SBC when using a fixed value of σ when simulating datasets for Bayesian t-test. A) Histories of log-gamma statistics for the model index, posterior mean (mu) and f(i, y) = (i− 1 2 )(sd(y)−1) (model_sd_m1), B) Calibration plot, C) and D) Separate calibration …
Figure 8
Figure 8. Figure 8: Overall distribution of posterior model probabilities A) for scenarios where [PITH_FULL_IMAGE:figures/full_fig_p037_8.png]
Figure 9
Figure 9. Figure 9: Convergence of the good check when a single datapoint is simulated from the [PITH_FULL_IMAGE:figures/full_fig_p040_9.png]
Figure 10
Figure 10. Figure 10: ECDF difference plots from SBC (A) and binary calibration plots (B, C) for [PITH_FULL_IMAGE:figures/full_fig_p041_10.png]
Figure 11
Figure 11. Figure 11: Results of SBC when using a πprior(σ 2 ) = 1 1+σ2 when simulating datasets for Bayesian t-test. A) ECDF difference plot for the derived quantity f(i, y) = (i − 1 2 )(sd(y) − 1) (model_sd_m1). B) and C) Separate binary prediction calibration plots for datasets with hig…
Figure 12
Figure 12. Figure 12: Results of SBC when using a σ ∼ HalfCauchy(0, 1) when simulating datasets for Bayesian t-test. A) ECDF difference plot for the derived quantity f(i, y) = (i − 1 2 )(sd(y) − 1) (model_sd_m1). B) and C) Separate binary prediction calibration plots for datasets with high…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 25 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type url volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.all := #1 'mid.sentence := ...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Bürkner, P.-C. (2017). brms: An R Package for Bayesian Multilevel Models Using Stan. Journal of Statistical Software\/ , 80(1): 1–28. ://www.jstatsoft.org/index.php/jss/article/view/v080i01

    work page 2017

  4. [4]

    D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., and Riddell, A

    Carpenter, B., Gelman, A., Hoffman, M. D., Lee, D., Goodrich, B., Betancourt, M., Brubaker, M., Guo, J., Li, P., and Riddell, A. (2017). Stan: A Probabilistic Programming Language. Journal of Statistical Software\/ , 76(1): 1–32. ://www.jstatsoft.org/index.php/jss/article/view/v076i01

    work page 2017

  5. [5]

    Dickey, J. M. and Lientz, B. P. (1970). The Weighted Likelihood Ratio, Sharp Hypotheses about Chances, the Order of a Markov Chain. The Annals of Mathematical Statistics\/ , 41(1): 214--226. ://www.jstor.org/stable/2239734

    work page arXiv 1970

  6. [6]

    Dimitriadis, T., Dümbgen, L., Henzi, A., Puke, M., and Ziegel, J. (2022). Honest calibration assessment for binary outcome predictions. Biometrika\/ , 110(3): 663--680. ://doi.org/10.1093/biomet/asac068

    doi:10.1093/biomet/asac068 2022

  7. [7]

    Dimitriadis, T., Gneiting, T., and Jordan, A. I. (2021). Stable reliability diagrams for probabilistic classifiers. Proceedings of the National Academy of Sciences\/ , 118(8): e2016191118. ://www.pnas.org/doi/abs/10.1073/pnas.2016191118

    doi:10.1073/pnas.2016191118 2021

  8. [8]

    Primed Priors for Simulation-Based Validation of Bayesian Models

    Fazio, L., Scholz, M., Aguilar, J. E., and Bürkner, P.-C. (2025). Primed Priors for Simulation-Based Validation of Bayesian Models. ://arxiv.org/abs/2408.06504

    work page Pith review arXiv 2025

Show all 38 references
  1. [9]

    Gaffke, N. (2005). Three test statistics for a nonparametric one-sided hypothesis on the mean of a nonnegative variable. Mathematical Methods of Statistics\/ , 14(4): 451--467

    2005

  2. [10]

    Gelman, A., Greengard, P., Gershunskaya, J., Savitsky, T., and Goodrich, B. (2023). Understanding posterior recalibration for a simple example. Unpublished manuscript.\/ . ://sites.stat.columbia.edu/gelman/research/unpublished/calibration_toy.pdf

    2023

  3. [11]

    C., Carpenter, B., Yao, Y., Kennedy, L., Gabry, J., Bürkner, P.-C., and Modrák, M

    Gelman, A., Vehtari, A., Simpson, D., Margossian, C. C., Carpenter, B., Yao, Y., Kennedy, L., Gabry, J., Bürkner, P.-C., and Modrák, M. (2020). Bayesian Workflow. ://arxiv.org/abs/2011.01808

    2020 arXiv

  4. [12]

    Geweke, J. (2004). Getting it right. Journal of the American Statistical Association\/ , 99: 799--804

    2004

  5. [13]

    F., Singmann, H., and Wagenmakers, E.-J

    Gronau, Q. F., Singmann, H., and Wagenmakers, E.-J. (2020). bridgesampling : An R Package for Estimating Normalizing Constants. Journal of Statistical Software\/ , 92(10): 1--29

    2020

  6. [14]

    Hartmann, M., Agiashvili, G., B\" u rkner, P., and Klami, A. (2020). Flexible Prior Elicitation via the Prior Predictive Distribution. In Peters, J. and Sontag, D. (eds.), Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI)\/ , volume 124 of Proc...

    2020

  7. [15]

    G., Fuglstad, G.-A., and Riebler, A

    Hem, I. G., Fuglstad, G.-A., and Riebler, A. (2022). makemyprior: Intuitive Construction of Joint Priors for Variance Parameters in R. ://arxiv.org/abs/2105.09712

    2022 arXiv

  8. [16]

    T., Pericchi, L

    Key, J. T., Pericchi, L. R., and Smith, A. F. M. (1999). Bayesian model choice: What and why. In Bernardo, J. M., Berger, J. O., Dawid, A. P., and Smith, A. F. M. (eds.), Bayesian Statistics 6:Proceedings of the Sixth Valencia International Meeting\/ , 343--370

    1999

  9. [17]

    H., Modrák, M., and Säilynoja, T

    Kim, S., Moon, A. H., Modrák, M., and Säilynoja, T. (2022). SBC: Simulation based calibration for rstan/cmdstanr models. ://github.com/hyunjimoon/SBC/

    2022

  10. [18]

    and Thomas, P

    Learned-Miller, E. and Thomas, P. S. (2020). A New Confidence Interval for the Mean of a Bounded Random Variable. ://arxiv.org/abs/1905.06208

    2020 arXiv

  11. [19]

    Liu, C. C. and Aitkin, M. (2008). Bayes factors: Prior sensitivity and model generalizability. Journal of Mathematical Psychology\/ , 52(6): 362--375. ://www.sciencedirect.com/science/article/pii/S002224960800028X

    2008

  12. [20]

    and Schilling, S

    Meng, X.-L. and Schilling, S. (2002). Warp bridge sampling. Journal of Computational and Graphical Statistics\/ , 11(3): 552--586

    2002

  13. [21]

    H., Kim, S., B \"u rkner, P., Huurre, N., Faltejskov \'a , K., Gelman, A., and Vehtari, A

    Modr \'a k, M., Moon, A. H., Kim, S., B \"u rkner, P., Huurre, N., Faltejskov \'a , K., Gelman, A., and Vehtari, A. (2025). Simulation-Based Calibration Checking for Bayesian Computation: The Choice of Test Quantities Shapes Sensitivity . Bayesian Analysis\/ , 20(2): 461 -- 48...

    2025 doi

  14. [22]

    Morey, R. D. and Rouder, J. N. (2024). BayesFactor: Computation of Bayes Factors for Common Designs\/ . R package version 0.9.12-4.7. ://CRAN.R-project.org/package=BayesFactor

    2024

  15. [23]

    and Barber, J

    Ogle, K. and Barber, J. J. (2020). Ensuring identifiability in hierarchical mixed effects Bayesian models. Ecological Applications\/ , 30(7): e02159. ://esajournals.onlinelibrary.wiley.com/doi/abs/10.1002/eap.2159

    2020 doi

  16. [24]

    a ilynoja, T., B\

    S\" a ilynoja, T., B\" u rkner, P.-C., and Vehtari, A. (2022). Graphical Test for Discrete Uniformity and Its Applications in Goodness-of-Fit Evaluation and Multiple Sample Comparison. Statistics and Computing\/ , 32(2). ://doi.org/10.1007/s11222-022-10090-6

    2022 doi

  17. [25]

    J., Modrák, M., and Vasishth, S

    Schad, D. J., Modrák, M., and Vasishth, S. (2025). How accurate are Bayes factor-based null hypothesis tests? A simulation study. ://arxiv.org/abs/2406.08022

    2025 arXiv

  18. [26]

    J., Nicenboim, B., Bürkner, P.-C., Betancourt, M., and Vasishth, S

    Schad, D. J., Nicenboim, B., Bürkner, P.-C., Betancourt, M., and Vasishth, S. (2023). Workflow techniques for the robust use of bayes factors. Psychological Methods\/ , 28(6): 1404--1426. ://doi.apa.org/doi/10.1037/met0000472

    2023 doi

  19. [27]

    J., Nicenboim, B., and Vasishth, S

    Schad, D. J., Nicenboim, B., and Vasishth, S. (2024). Data aggregation can lead to biased inferences in Bayesian linear mixed models and Bayesian analysis of variance. Psychological Methods\/ . ://doi.apa.org/doi/10.1037/met0000621

    2024 doi

  20. [28]

    Sekulovski, N., Marsman, M., and Wagenmakers, E.-J. (2024). A Good check on the Bayes factor. Behavior Research Methods\/ , 56(8): 8552--8566. ://doi.org/10.3758/s13428-024-02491-4

    2024 doi

  21. [29]

    Säilynoja, T., Schmitt, M., Bürkner, P.-C., and Vehtari, A. (2025). Posterior SBC: Simulation-Based Calibration Checking Conditional on Data. ://arxiv.org/abs/2502.03279

    2025 arXiv

  22. [30]

    Talts, S., Betancourt, M., Simpson, D., Vehtari, A., and Gelman, A. (2020). Validating Bayesian inference algorithms with simulation-based calibration. ://www.stat.columbia.edu/ gelman/research/unpublished/sbc.pdf

    2020

  23. [31]

    and Okada, K

    Tsukamura, Y. and Okada, K. (2024). The ``neglecting the vectorization`` error in Stan: erroneous coding practices for computing marginal likelihood and Bayes factors in models with vectorized truncated distributions. Behaviormetrika\/ , 51(2): 635--644. ://doi.org/10.1007/s41...

    2024 doi

  24. [32]

    and Ly, A

    Wagenmakers, E.-J. and Ly, A. (2023). History and nature of the Jeffreys--Lindley paradox. Archive for History of Exact Sciences\/ , 77(1): 25--72. ://doi.org/10.1007/s00407-022-00298-3

    2023 doi

  25. [33]

    F., S m \'i ra, M., Epskamp, S., Matzke, D., Rouder, J

    Wagenmakers, E.-J., Marsman, M., Jamil, T., Ly, A., Verhagen, J., Love, J., Selker, R., Gronau, Q. F., S m \'i ra, M., Epskamp, S., Matzke, D., Rouder, J. N., and Morey, R. D. (2018). Bayesian inference for psychology. Part I: Theoretical advantages and practical ramifications...

    2018 doi

  26. [34]

    and Zhao, L

    Wang, W. and Zhao, L. H. (2003). Nonparametric tests for the mean of a non-negative population. Journal of Statistical Planning and Inference\/ , 110(1): 75--96. ://www.sciencedirect.com/science/article/pii/S0378375801002944

    2003

  27. [35]

    D., and Reich, B

    Yanchenko, E., Bondell, H. D., and Reich, B. J. (2021). The R2D2 Prior for Generalized Linear Mixed Models. ://arxiv.org/abs/2111.10718

    2021 arXiv

  28. [36]

    and Domke, J

    Yao, Y. and Domke, J. (2023). Discriminative Calibration: Check Bayesian Computation from Simulations and Flexible Classifier. In Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., and Levine, S. (eds.), Advances in Neural Information Processing Systems\/ , volume 36, ...

    2023

  29. [37]

    Yao, Y., Vehtari, A., Simpson, D., and Gelman, A. (2018). Using Stacking to Average Bayesian Predictive Distributions (with Discussion) . Bayesian Analysis\/ , 13(3): 917 -- 1007. ://doi.org/10.1214/17-BA1091

    2018 doi

  30. [38]

    J., Tran, M.-N., and Klein, N

    Yu, X., Nott, D. J., Tran, M.-N., and Klein, N. (2021). Assessment and adjustment of approximate inference algorithms using the law of total variance. Journal of Computational and Graphical Statistics\/ , 30: 977--990. ://doi.org/10.1080/10618600.2021.1880921

    2021

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.