Pith. sign in

REVIEW 3 major objections 5 minor 157 references

The Curious Case of the Default Settings: Evaluating Default Performance of Variational Inference Software

T0 review · 3 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Default VI settings can silently bias posterior estimates in major Python packages, even for simple one-dimensional conjugate models.

desk verdict A useful, well-scoped empirical study of VI defaults, but the headline 5% bias result doesn't actually use the default iteration count—worth engaging, but needs revision. read the letter →

arxiv 2608.01403 v1 pith:7DAFIY7Q submitted 2026-08-02 stat.CO

Madelyn Andersen , Elizabeth Bersson , Tamara Broderick This is my paper
classification stat.CO
keywords variationalinferencedefaultsettingsprobabilisticprogrammingPyMCNumPyroTensorFlowProbabilityposteriorapproximationADVI
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper evaluates what happens when a typical user runs variational inference without touching any settings in three widely used Python libraries—PyMC, NumPyro, and TensorFlow Probability. Using simple one-dimensional conjugate models with known posterior distributions, it shows that defaults are not merely suboptimal; they can be systematically wrong. PyMC's default single Monte Carlo gradient sample plus its Adagrad_Window optimizer produces a posterior standard deviation that is biased about 5% high for millions of iterations, while TensorFlow Probability can fail silently, returning nonsensical results, when a constrained parameter is left untransformed. The paper argues these failures are implementation-level, not intrinsic to variational inference, and shows that switching to Adam and using 100 Monte Carlo samples removes the bias. The practical stake: practitioners who rely on defaults may report posterior summaries that are quietly wrong, without any error message.

What carries the argument

The evaluation machinery is a one-dimensional Gaussian–Gaussian conjugate model with known variance, for which the posterior is exactly Gaussian and the variational optimum is exactly the true posterior, giving an analytic ground truth for the posterior mean and standard deviation. The paper tracks the variational parameters (posterior mean and standard deviation) over up to 1.6 million iterations under each package's defaults and under non-default settings, comparing against this ground truth. This isolates the effect of software implementation choices—Monte Carlo gradient sample count, optimizer (Adagrad_Window vs. Adam), initialization scheme, and support transformations—from the intrinsi

What would settle it

Run PyMC's default ADVI on a two-dimensional or non-conjugate model where the posterior can be computed exactly or via a long MCMC chain, and check whether the posterior standard deviation still sits roughly 5% off after millions of iterations; if the bias disappears when the model is no longer conjugate, the benchmark-specific explanation fails.

Watch

Extended reading notes

Core claim

The central discovery is that, on a benchmark where the posterior is exactly Gaussian and the best variational approximation is exactly the true posterior, software defaults can nonetheless introduce systematic error. In PyMC, with the default single Monte Carlo sample and Adagrad_Window optimizer, the variational posterior's standard deviation overshoots the analytic value by roughly 5% and stays there for millions of iterations; increasing the Monte Carlo sample size to 100 or switching to the Adam optimizer removes the bias. NumPyro and TFP, which default to Adam, show no such bias under the same benchmark, though their trajectories oscillate and improve with more Monte Carlo samples. The

Load-bearing premise

The paper's recommendations assume that failure modes observed in one-dimensional conjugate models—where the variational family contains the true posterior—will also appear in realistic non-conjugate, multi-dimensional models, which the paper itself notes are the common case.

Editorial extensions

If this is right

  • In PyMC, changing the default optimizer from Adagrad_Window to Adam, or raising the Monte Carlo sample size to 100, removes the roughly 5% bias in the posterior standard deviation.
  • All three packages produce smaller oscillations in the variational trajectories when the number of Monte Carlo gradient samples is increased from 1 to 100.
  • A TFP user who fits a model with constrained support (e.g., a beta prior) without adding a bijector can get a NaN ELBO and a meaningless posterior approximation, with no error raised.
  • Initialization matters: in a bimodal example, different random restarts lead the variational approximation to different local modes, so multiple restarts are recommended in multimodal settings.
  • Following the paper's recommendations—prefer NumPyro, use 100 MC samples, switch PyMC to Adam, and verify support-matching transformations—makes the one-dimensional benchmarks behave predictably.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A reasonable next test is whether the 5% bias, driven by the interaction of a single noisy gradient sample with Adagrad_Window's step-size schedule, reappears in higher-dimensional or ill-conditioned models, where gradient noise is larger; the paper does not test this.
  • The TFP silent-failure mode suggests that a cheap, model-agnostic diagnostic—monitoring the ELBO for NaN and checking whether sampled variational draws lie inside the support—could catch most of these failures before any summary is reported.
  • The paper's preference for NumPyro rests on its conjugate ELBO estimator (TraceMeanField_ELBO); in non-conjugate models NumPyro switches to a different estimator (Trace_ELBO), so the default-experience ranking could change outside the conjugate setting.
  • One natural extension would be to run the same default-versus-tuned comparison on a small non-conjugate model where MCMC is affordable, and check whether the bias or silent failure appears in any package.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper evaluates the default variational inference (VI) settings of PyMC, NumPyro, and TensorFlow Probability on simple analytic conjugate models, with the one-dimensional Gaussian–Gaussian model as the primary testbed. Using the known posterior as ground truth, the authors report that PyMC's default settings (one Monte Carlo gradient sample and Adagrad_Window) exhibit a persistent bias of about 5% in the approximate posterior standard deviation, while NumPyro and TFP show only minor oscillations. They further examine how initialization control, transformation handling, and diagnostic monitoring differ across the packages, including a silent-failure mode for TFP when a required transformation is omitted. The paper concludes with practical guidance for users and developers, recommending increases to the MC sample count, use of Adam optimizers, and explicit verification of support transformations.

Significance. If the central claims are robust, the paper provides a useful empirical audit of off-the-shelf VI software, a topic that is often neglected relative to methodological advances. The benchmark design is well suited to isolating software-level effects: the Gaussian target is exactly in the variational family, so discrepancies from the analytic posterior can be attributed to implementation choices rather than variational-family limitations. The paper fixes software versions, states default hyperparameters, and makes demo code available. These are concrete strengths. The main risk is that the headline finding rests on a non-default iteration count and a single trajectory, and that the practical recommendations outrun the evidence presented.

major comments (3)
  1. [§3.1.1, §3.1.3, Figure 1] The central quantitative claim is that PyMC's default settings produce a systematic ~5% bias in the approximate posterior standard deviation. The supporting runs use a manually specified 1.6 million iterations (§3.1.3), whereas a genuinely default PyMC run terminates at 10,000 iterations when the number of iterations is not specified (§3.1.1). The manuscript never reports the approximate σ_p at iteration 10,000 for the default 1-MC/Adagrad_Window configuration, nor whether the biased plateau has already been reached by the default stopping time. The abstract's conclusion that 'default settings can yield biased approximations' therefore conflates the optimizer and MC-sample defaults with a non-default iteration budget. Please report the trajectory and the bias at the default stopping time, or explicitly reframe the claim as a long-run property of the default optimizer/MC settings under a
  2. [§3.1.2, Figure 1] The 5% bias result is demonstrated with a single trajectory and no error bars or multiple-seed analysis. Stochastic variational inference trajectories depend on random number generation, and the text itself describes oscillations in the same runs. Without repeated runs, 'systematic' and 'persists for millions of iterations' are not established. Please add multiple seeds (e.g., 10 or more) for the PyMC default configuration and report the distribution of final σ_p values, or otherwise provide quantitative evidence that the bias is reproducible rather than a single-run artifact.
  3. [§4.5 and §5.3.1] The practical guide recommends preferring NumPyro and avoiding PyMC and TFP for default users, based on the one-dimensional conjugate benchmarks. Section 5.3.1 itself acknowledges that most applied models are non-conjugate and multi-dimensional, and that NumPyro's TraceMeanField_ELBO estimator for conjugate cases differs from the Trace_ELBO used for non-conjugate models. No evidence is provided that the observed failure modes transfer to realistic models. The recommendations should either be explicitly scoped to the tested model class or supplemented with experiments in non-conjugate and higher-dimensional settings; otherwise the guide may mislead practitioners applying the defaults to problems outside this narrow benchmark family.
minor comments (5)
  1. [§3.1.2] The text says 'our preliminary further experiments suggest that the bias does persist' for beta, inverse-gamma, and gamma models, but no results are shown. Either include these results in an appendix or remove the preliminary claim, since it currently functions as unsupported evidence.
  2. [§2.2] The hyperparameters for the one-dimensional Gaussian–Gaussian benchmark (prior mean/variance, likelihood variance, sample size, and data) are not stated in the main text. Please provide the exact values so the experiments are reproducible without consulting the code repository.
  3. [§3.1.1] The definition of 'default' for NumPyro and TFP should be stated more carefully: both packages require the user to supply an optimizer and an iteration count, so the use of Adam is a convention adopted from official vignettes rather than a package default. This distinction is important for interpreting the comparison.
  4. [§3.1.3] The sentence 'With the exception of Figure 2...' is slightly ambiguous. It would be clearer to state explicitly in each figure caption that all runs use a manually specified 1.6 million iterations, and to specify whether the PyMC Adam run in Figure 2 uses the same iteration count.
  5. [Throughout] There is inconsistent terminology: 'systemic bias' appears in §3.1.2 while 'systematic bias' is used elsewhere. Please standardize.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's claims are benchmarked against external analytic ground truths, not against its own outputs.

full rationale

The paper evaluates VI software by comparing variational trajectories to closed-form posterior means and variances of one-dimensional conjugate models, especially the Gaussian-Gaussian model. These analytic targets are external to the software and to the paper's own derivations, so success is defined by an independent ground truth rather than by the software's own outputs. The 'best variational approximation' is the ELBO maximizer, which in the Gaussian-Gaussian case coincides with the exact posterior because the posterior lies in the variational family; this is a mathematical fact, not a fitted result. Claims about a ~5% bias in PyMC, oscillations in NumPyro/TFP, mitigation by more MC samples or Adam, and TFP's silent failure without transforms are direct empirical observations, not predictions derived from fitted parameters. Self-citations (e.g., Giordano et al.) are used for background context on known VI limitations and are not load-bearing for the paper's central quantitative findings. The paper's limitation section 5.3.1 explicitly acknowledges that most applied models are non-conjugate and multidimensional, and that findings may not transfer; this is an honest scope restriction, not circular reasoning. The skeptical concern that the bias is only shown at a manually specified 1.6 million iterations rather than PyMC's 10,000-iteration default is a validity/generalizability caveat about whether the default termination point exhibits the bias, but it does not constitute a definitional or fitted-input circularity under the specified criteria. Hence no significant circularity is present.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new math or entities. It evaluates existing software against analytic benchmarks. The main assumptions are that the analytic ground truths apply and that the package defaults are as described. The paper's own limitations acknowledge the generality assumption.

assumptions (4)
  • standard math Standard VI theory: ELBO equivalence, mean-field family, reverse KL.
    Used in Section 2.1 to define the optimization problem and best variational approximation.
  • standard math Analytic conjugate posterior formulas for Gaussian-Gaussian, Beta-Binomial, Gamma-Poisson, etc.
    Used as ground truth in Section 2.2 and Appendix A.
  • domain assumption Package documentation and source code accurately describe default behavior.
    They inspect PyMC, NumPyro, TFP source/docs (Section 3.1.1) to set defaults; if versions change or docs are misleading, the results may not reflect current software.
  • domain assumption One-dimensional conjugate benchmarks are informative for general VI behavior.
    Section 2.2 states failures in one dimension are expected to worsen in higher dimensions; this is plausible but unproven. Section 5.3.1 is the limitation.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Curious Case of the Default Settings: Evaluating Default Performance of Variational Inference Software." pith.science (2026). https://pith.science/paper/7DAFIY7Q

@misc{pith2026260801403,
  author       = {Pith},
  title        = {Pith review of: The Curious Case of the Default Settings: Evaluating Default Performance of Variational Inference Software},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7DAFIY7Q}},
  note         = {Machine review of arXiv:2608.01403}
}
read the original abstract

We systematically evaluate a suite of off-the-shelf variational inference (VI) software packages from the perspective of a standard practitioner. Using simple analytic benchmark models, we assess the accuracy and stability of the default VI settings in PyMC, NumPyro, and TensorFlow Probability. Unlike previous research focusing on methodological advances, our evaluation emphasizes software implementation and the default configurations that typical users encounter. Our results show that default settings can yield biased approximations of posterior summaries even for simple one-dimensional conjugate models, controls of initialization and transformations differ between software implementations, and relying on defaults may yield silent failures or poor approximations.

Figures

Figures reproduced from arXiv: 2608.01403 by the authors.

Figure 1
Figure 1. Default vs. tuned Monte Carlo gradient samples in PyMC (one-dimensional Gaussian, known variance) run with PyMC’s default Adagrad_Window optimizer. The top and bottom figures plot the variational trajectories under default (1) and adjusted (100) MC sample values for the posterior parameters σp and µp, respectively, where trajectories of the same color between plots are derived from the same VI run. Upper plot: Using… view at source ↗
Figure 2
Figure 2. Default vs. tuned Monte Carlo gradient samples in PyMC (one-dimensional Gaussian, known variance) run with Adam optimizer and default Adam settings. The upper and lower figures plot the variational trajectories under default (1) and adjusted (100) MC sample values for the posterior parameters σp and µp, respectively, where trajectories of the same color between plots are derived from the same VI run. Upper plot: Usi… view at source ↗
Figure 3
Figure 3. Default vs. tuned Monte Carlo gradient samples in NumPyro (one-dimensional Gaussian, known variance). The top and bottom figures plot the variational trajectories under default (1) and adjusted (100) MC sample values for the posterior parameters σp and µp, respectively, where trajectories of the same color between plots are derived from the same VI run. Upper plot: Using a single MC sample oscillates with a small ma… view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: Default vs. tuned Monte Carlo gradient samples in TFP (one-dimensional Gaussian, known variance). The top and bottom figures plot the variational trajectories under default (1) and adjusted (100) MC sample values for the posterior parameters σp and µp, respectively, wh…
Figure 5
Figure 5. Figure 5: The true target density (black solid line) is a balanced mixture of beta(20, 2) and beta(2, 20), producing two narrow modes. Shown are variational approximations (dashed lines) obtained by fitting a Gaussian in the unconstrained space and mapping it to (0, 1) via a sig…
Figure 6
Figure 6. Figure 6: Support mismatch without a transform. The untransformed fit drifts out of support. Adding the sigmoid bijector yields a valid fit. The inset axes zoom out on the fitted VI distributions after 5,000 and 100,000 iterations to show behavior not visible on the smaller scal…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

157 extracted references · 70 canonical work pages

  1. [1]

    and Nero, Anthony V

    Price, Phillip N. and Nero, Anthony V. and Gelman, Andrew , journal =. Bayesian Prediction of Mean Indoor Radon Concentrations for. 1996 , volume =

    1996

  2. [2]

    2025 , booktitle =

    Fonnesbeck, Christopher and Abril, Oriol and Andorra, Alex and Carroll, Colin and Reynaldo, Farhan , title =. 2025 , booktitle =

    2025

  3. [3]

    Scientific Reports , volume =

    Bayesian Hierarchical Model Predicts Biopharmaceutical Stability Indicators and Shelf Life with Application to Multivalent Human Papillomavirus Vaccine , author =. Scientific Reports , volume =

  4. [4]

    How to increase the number of dropped evaluations variational bayes cmdstanr? , year =

  5. [5]

    2021 , month = nov, howpublished =

    Yao, Yuling , title =. 2021 , month = nov, howpublished =

    2021

  6. [6]

    Khazaei, Yeganeh and K. Using a. Scientific Reports , volume =

  7. [7]

    James and Gilliland, Frank D

    Weng, Jingying and Molshatzki, Noa and Marjoram, Paul and Gauderman, W. James and Gilliland, Frank D. and Eckel, Sandrah P. , year = 2023, month = apr, journal =. Longitudinal Hierarchical

    2023

  8. [8]

    and Petkova, Eva and Park, Hyung G

    Wu, Danni and Goldfeld, Keith S. and Petkova, Eva and Park, Hyung G. , year = 2024, month = sep, journal =. A

    2024

Show all 157 references
  1. [9]

    Proceedings of the 21st International Congress on Modelling and Simulation (MODSIM) , pages=

    Using the Bayesian Logistic Regression Model to Determine the Relationship of Demographics and Hyperaldosteronism , author=. Proceedings of the 21st International Congress on Modelling and Simulation (MODSIM) , pages=

  2. [10]

    Chinese Journal of Epidemiology , volume=

    Study on the interaction under logistic regression modeling , author=. Chinese Journal of Epidemiology , volume=. 2008 , pmid=

    2008

  3. [11]

    Political Science Research and Methods , volume=

    Compression and Conditional Effects: A Product Term Is Essential When Using Logistic Regression to Test for Interaction , author=. Political Science Research and Methods , volume=. 2016 , publisher=

    2016

  4. [12]

    PeerJ , volume=

    Bayesian estimation of the measurement of interactions in epidemiological studies , author=. PeerJ , volume=. 2024 , publisher=

    2024

  5. [13]

    and Luhmann, Christian , title =

    Orduz, Juan and Vincent, Benjamin T. and Luhmann, Christian , title =. 2023 , booktitle =

    2023

  6. [14]

    Journal of Machine Learning Research , year =

    Yuling Yao and Aki Vehtari and Andrew Gelman , title =. Journal of Machine Learning Research , year =

  7. [15]

    Madelyn Andersen , title =

  8. [16]

    , institution =

    Murphy, Kevin P. , institution =. Conjugate

  9. [17]

    Kasprzak and Ryan Giordano and Tamara Broderick , title =

    Mikolaj J. Kasprzak and Ryan Giordano and Tamara Broderick , title =. Journal of Machine Learning Research , year =

  10. [18]

    , month = jun, year =

    Hoff, Peter D. , month = jun, year =. A

  11. [19]

    Journal of Machine Learning Research , year =

    Lu Zhang and Bob Carpenter and Andrew Gelman and Aki Vehtari , title =. Journal of Machine Learning Research , year =

  12. [20]

    arXiv preprint arXiv:2307.07849 , author =

    Variational. arXiv preprint arXiv:2307.07849 , author =

    arXiv

  13. [21]

    Blei , title =

    Alp Kucukelbir and Dustin Tran and Rajesh Ranganath and Andrew Gelman and David M. Blei , title =. Journal of Machine Learning Research , year =

  14. [22]

    and Trippe, Brian L

    Nguyen, Tin D. and Trippe, Brian L. and Broderick, Tamara , booktitle =. Many processors, little time:. 2022 , volume =

    2022

  15. [23]

    arXiv preprint arXiv:2210.00580 , author =

    arXiv

  16. [24]

    and Robin, S

    Latouche, P. and Robin, S. , year =. Variational. Statistics and Computing , volume =

  17. [25]

    Proceedings of the 2016 ICML Workshop on \#Data4Good: Machine Learning in Social Good Applications , year=

    Fast robustness quantification with variational Bayes , author=. Proceedings of the 2016 ICML Workshop on \#Data4Good: Machine Learning in Social Good Applications , year=

    2016

  18. [26]

    and Malkin, Nikolay and Jain, Moksh and Everett, Katie and Graikos, Alexandros and Bengio, Yoshua , year =

    Hu, Edward J. and Malkin, Nikolay and Jain, Moksh and Everett, Katie and Graikos, Alexandros and Bengio, Yoshua , year =. Advances in Neural Information Processing Systems , volume =

  19. [27]

    and Ghahramani, Zoubin and Jaakkola, Tommi S

    Jordan, Michael I. and Ghahramani, Zoubin and Jaakkola, Tommi S. and Saul, Lawrence K. , year =. An. Machine Learning , volume =

  20. [28]

    Jordan , title =

    Ryan Giordano and Tamara Broderick and Michael I. Jordan , title =. Journal of Machine Learning Research , year =

  21. [29]

    , year =

    Kucukelbir, Alp and Ranganath, Rajesh and Gelman, Andrew and Blei, David M. , year =. Automatic. Advances in Neural Information Processing Systems , volume =

  22. [30]

    Statistical Science , author =

    Past,. Statistical Science , author =. 2024 , pages =

    2024

  23. [31]

    Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume =

    Rue, Håvard and Martino, Sara and Chopin, Nicolas , title =. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume =

  24. [32]

    Hoffman and David M

    Matthew D. Hoffman and David M. Blei and Chong Wang and John Paisley , title =. Journal of Machine Learning Research , year =

  25. [33]

    2014 , editor =

    Ranganath, Rajesh and Gerrish, Sean and Blei, David , booktitle =. 2014 , editor =

    2014

  26. [34]

    Yes, but

    Yao, Yuling and Vehtari, Aki and Simpson, Daniel and Gelman, Andrew , year =. Yes, but. Proceedings of the International Conference on Machine Learning , pages =

  27. [35]

    and Kucukelbir, Alp and McAuliffe, Jon D

    Blei, David M. and Kucukelbir, Alp and McAuliffe, Jon D. , year =. Variational. Journal of the American Statistical Association , volume =. 1601.00670 , primaryclass =

    arXiv

  28. [36]

    Variational

    Rezende, Danilo Jimenez and Mohamed, Shakir , year =. Variational. Proceedings of the International Conference on Machine Learning , pages =

  29. [37]

    and Salimans, Tim and Jozefowicz, Rafal and Chen, Xi and Sutskever, Ilya and Welling, Max , year =

    Kingma, Diederik P. and Salimans, Tim and Jozefowicz, Rafal and Chen, Xi and Sutskever, Ilya and Welling, Max , year =. Improving. Advances in Neural Information Processing Systems , volume =

  30. [38]

    Proceedings of the 39th International Conference on Machine Learning , pages =

    Fat--Tailed Variational Inference with Anisotropic Tail Adaptive Flows , author =. Proceedings of the 39th International Conference on Machine Learning , pages =. 2022 , volume =

    2022

  31. [39]

    Discover Social Science and Health , volume=

    Determining the risk factors of under-five morbidity in Bangladesh: a Bayesian logistic regression approach , author=. Discover Social Science and Health , volume=. 2023 , publisher=

    2023

  32. [40]

    Stan GitHub Repository , url =

    advi.hpp , month = feb, year =. Stan GitHub Repository , url =

  33. [41]

    2nd International Conference on Learning Representations (ICLR) , year=

    Auto-Encoding Variational Bayes , author=. 2nd International Conference on Learning Representations (ICLR) , year=

  34. [42]

    Foundations and Trends in Machine Learning , volume=

    An Introduction to Variational Autoencoders , author=. Foundations and Trends in Machine Learning , volume=

  35. [43]

    Journal of Machine Learning Research , year =

    Shakir Mohamed and Mihaela Rosca and Michael Figurnov and Andriy Mnih , title =. Journal of Machine Learning Research , year =

  36. [44]

    , title=

    Domke, Justin and Sheldon, Daniel R. , title=. Advances in Neural Information Processing Systems , volume=

  37. [45]

    Bayesian Analysis , year=

    Yao, Yuling and Vehtari, Aki and Simpson, Daniel and Gelman, Andrew , title=. Bayesian Analysis , year=

  38. [46]

    and Kasprzak, Mikołaj and Campbell, Trevor and Broderick, Tamara , title=

    Huggins, Jonathan H. and Kasprzak, Mikołaj and Campbell, Trevor and Broderick, Tamara , title=. Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS) , year=

  39. [47]

    2006 , month = oct, day =

    National Ambient Air Quality Standards for Particulate Matter; Final Rule , journal =. 2006 , month = oct, day =

    2006

  40. [48]

    Gelman, Andrew and Hill, Jennifer , title =

  41. [49]

    and Lee, Daniel and Goodrich, Ben and Betancourt, Michael and Brubaker, Marcus and Guo, Jiqiang and Li, Peter and Riddell, Allen , title=

    Carpenter, Bob and Gelman, Andrew and Hoffman, Matthew D. and Lee, Daniel and Goodrich, Ben and Betancourt, Michael and Brubaker, Marcus and Guo, Jiqiang and Li, Peter and Riddell, Allen , title=. Journal of Statistical Software , year=

  42. [50]

    and Fonnesbeck, Christopher , title=

    Salvatier, John and Wiecki, Thomas V. and Fonnesbeck, Christopher , title=. PeerJ Computer Science , year=

  43. [51]

    Chen and Martin Jankowiak and Fritz Obermeyer and Neeraj Pradhan and Theofanis Karaletsos and Rohit Singh and Paul Szerlip and Paul Horsfall and Noah D

    Eli Bingham and Jonathan P. Chen and Martin Jankowiak and Fritz Obermeyer and Neeraj Pradhan and Theofanis Karaletsos and Rohit Singh and Paul Szerlip and Paul Horsfall and Noah D. Goodman , title =. Journal of Machine Learning Research , year =

  44. [52]

    Journal of the American Statistical Association , author=

    Frequentist Consistency of Variational Bayes , volume=. Journal of the American Statistical Association , author=. 2019 , pages=

    2019

  45. [53]

    IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

    Zhang, Cheng and Butepage, Judith and Kjellstrom, Hedvig and Mandt, Stephan , title=. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume=

  46. [54]

    arXiv preprint arXiv:1705.10843 , year=

    Objective-Reinforced Generative Adversarial Networks (ORGAN) for Sequence Generation Models , author=. arXiv preprint arXiv:1705.10843 , year=

    arXiv

  47. [55]

    2017 , eprint=

    Edward: A library for probabilistic modeling, inference, and criticism , author=. 2017 , eprint=

    2017

  48. [56]

    Advances in Neural Information Processing Systems , volume=

    Liu, Qiang and Wang, Dilin , title=. Advances in Neural Information Processing Systems , volume=. 2016 , pages=

    2016

  49. [57]

    2024 , eprint=

    Theoretical Guarantees for Variational Inference with Fixed-Variance Mixture of Gaussians , author=. 2024 , eprint=

    2024

  50. [58]

    2020 , school =

    Jincheng Bai , title =. 2020 , school =

    2020

  51. [59]

    2016 , journal =

    Variational Inference for Medical Image Segmentation , author =. 2016 , journal =

    2016

  52. [60]

    npj Climate and Atmospheric Science , volume =

    Data-Driven Seasonal Climate Predictions via Variational Inference and Transformers , author =. npj Climate and Atmospheric Science , volume =

  53. [61]

    and Smith, Jared D

    Kavianihamedani, Hossein and Quinn, Julianne D. and Smith, Jared D. , journal=. New Diagnostic Assessment of MCMC Algorithm Effectiveness, Efficiency, Reliability, and Controllability , year=

  54. [62]

    Mathematics and Computers in Simulation (MATCOM) , author=

    Encoded. Mathematics and Computers in Simulation (MATCOM) , author=. 2022 , pages=

    2022

  55. [63]

    2020 , journal=

    Theoretical and Computational Guarantees of Mean Field Variational Inference for Community Detection , author=. 2020 , journal=

    2020

  56. [64]

    2021 , eprint=

    Challenges and Opportunities in High-dimensional Variational Inference , author=. 2021 , eprint=

    2021

  57. [65]

    2020 , eprint=

    Robust, Accurate Stochastic Optimization for Variational Inference , author=. 2020 , eprint=

    2020

  58. [66]

    2024 , eprint=

    Stable Training of Normalizing Flows for High-dimensional Variational Inference , author=. 2024 , eprint=

    2024

  59. [67]

    2024 , eprint=

    Preferential Normalizing Flows , author=. 2024 , eprint=

    2024

  60. [68]

    2024 , eprint=

    Disentangling impact of capacity, objective, batchsize, estimators, and step-size on flow VI , author=. 2024 , eprint=

    2024

  61. [69]

    and Teschendorff, A

    Ma, Z. and Teschendorff, A. E. , title =. Journal of Bioinformatics and Computational Biology , year =

  62. [70]

    Fay, R. E. and Herriot, R. A. , title =. Journal of the American Statistical Association , volume =

  63. [71]

    2008 , journal=

    Model-based Clustering of DNA Methylation Array Data: A Recursive-partitioning Algorithm for High-dimensional Data Arising as a Mixture of Beta Distributions , author=. 2008 , journal=

    2008

  64. [72]

    and Jordan, Michael I

    Jaakkola, Tommi S. and Jordan, Michael I. , journal =. A Variational Approach to. 1997 , editor =

    1997

  65. [73]

    Propagation Algorithms for Variational Bayesian Learning , volume =

    Ghahramani, Zoubin and Beal, Matthew , booktitle =. Propagation Algorithms for Variational Bayesian Learning , volume =

  66. [74]

    2010 , eprint=

    Graphical Comparison of MCMC Performance , author=. 2010 , eprint=

    2010

  67. [75]

    2025 , howpublished =

    Bob Carpenter and Madelyn Andersen and Aki Vehtari , title =. 2025 , howpublished =

    2025

  68. [76]

    2022 , howpublished =

    Vehtari, Aki , title =. 2022 , howpublished =

    2022

  69. [77]

    Variational Inference , year =

  70. [78]

    2020 , month =

    Dablander, Fabian , title =. 2020 , month =

    2020

  71. [79]

    Variational Inference in Stan , year =

  72. [80]

    2023 , note=

    Why and how Variational Inference underestimates variance? , author=. 2023 , note=

    2023

  73. [81]

    2023 , note=

    Parameters way off with variational inference, not sure why , author=. 2023 , note=

    2023

  74. [82]

    and Welling, Max , title =

    Salimans, Tim and Kingma, Diederik P. and Welling, Max , title =. 2015 , booktitle =

    2015

  75. [83]

    , title =

    Meyn, Sean and Tweedie, Richard L. , title =. 2009 , edition =

    2009

  76. [84]

    2018 , note=

    Question on variational inference , author=. 2018 , note=

    2018

  77. [85]

    2022 , note=

    Issue \#6271: Improve advice / diagnostics when VI gives terrible results , author=. 2022 , note=

    2022

  78. [86]

    2024 , howpublished=

    10 Robust Variational Inference Methods for Bayesian Networks , author=. 2024 , howpublished=

    2024

  79. [87]

    Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics , pages=

    A Theoretical Case Study of Structured Variational Inference for Community Detection , author=. Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics , pages=. 2020 , volume=

    2020

  80. [88]

    2022 , eprint=

    Bambi: A simple interface for fitting Bayesian linear models in Python , author=. 2022 , eprint=

    2022

  81. [89]

    , title =

    Bishop, Christopher M. , title =. 2006 , isbn =

    2006

  82. [90]

    2021 , journal =

    Stochastic Gradient Markov Chain Monte Carlo , author =. 2021 , journal =. https://doi.org/10.1080/01621459.2020.1847120 , pages =

    2021

  83. [91]

    and Rosenthal, Jeffrey S

    Roberts, Gareth O. and Rosenthal, Jeffrey S. , year=. General state space Markov chains and MCMC algorithms , volume=. Probability Surveys , pages =

  84. [92]

    Bayesian Modeling of Traffic-Related Air Pollutants:

    Ning, Yihong and Sun, Ruixiao and Hitchcock, David and Comert, Gurcan and Chen, Yuche , year =. Bayesian Modeling of Traffic-Related Air Pollutants:. Atmospheric Environment: X , volume =

  85. [93]

    Korean Journal of Anesthesiology , volume =

    Kwak, Sang Gyu and Kim, Jong Hae , title =. Korean Journal of Anesthesiology , volume =. 2017 , month = apr, pmid =

    2017

  86. [94]

    The Annals of Mathematical Statistics , volume =

    Herbert Robbins and Sutton Monro , title =. The Annals of Mathematical Statistics , volume =

  87. [95]

    2025 , note =

    Stan User's Guide , author =. 2025 , note =

    2025

  88. [96]

    2025 , howpublished =

    Bambi: Bayesian Model-Building Interface , author =. 2025 , howpublished =

    2025

  89. [97]

    2024 , howpublished =

    pypistats , author =. 2024 , howpublished =

    2024

  90. [98]

    Speagle , year=

    Joshua S. Speagle , year=. A Conceptual Introduction to. 1909.12313 , archivePrefix=

    1909 arXiv

  91. [99]

    Robert and George Casella , title =

    Christian P. Robert and George Casella , title =. 2005 , address =

    2005

  92. [100]

    Du Phan and Neeraj Pradhan and Martin Jankowiak , booktitle=

  93. [101]

    2017 , eprint=

    TensorFlow Distributions , author=. 2017 , eprint=

    2017

  94. [102]

    12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16) , year =

    Mart. 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16) , year =

  95. [103]

    2019 , eprint=

    Convergence Rates of Variational Posterior Distributions , author=. 2019 , eprint=

    2019

  96. [104]

    2012 , eprint=

    Convergence and asymptotic normality of variational Bayesian approximations for exponential family models with missing values , author=. 2012 , eprint=

    2012

  97. [105]

    Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics , pages =

    Proximity Variational Inference , author =. Proceedings of the Twenty-First International Conference on Artificial Intelligence and Statistics , pages =. 2018 , editor =

    2018

  98. [106]

    2019 , version =

    Source code for numpyro.infer.elbo , author =. 2019 , version =

    2019

  99. [107]

    2018 , month =

    Default value used for obj\_n\_mc in ADVI , author =. 2018 , month =

    2018

  100. [108]

    Source code for pymc.variational.approximations , author =

  101. [109]

    2023 , month =

    How to initialize ADVI , author =. 2023 , month =

    2023

  102. [110]

    Mathematical Programming , year=

    Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization , author=. Mathematical Programming , year=

  103. [111]

    Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics , pages =

    Inadequacy of interval estimates corresponding to variational Bayesian approximations , author =. Proceedings of the Tenth International Workshop on Artificial Intelligence and Statistics , pages =. 2005 , editor =

    2005

  104. [112]

    , title =

    Giordano, Ryan and Broderick, Tamara and Jordan, Michael I. , title =. Advances in Neural Information Processing Systems , volume =

  105. [113]

    Globally Convergent Variational Inference , volume =

    McNamara, Declan and Loper, Jackson and Regier, Jeffrey , booktitle =. Globally Convergent Variational Inference , volume =

  106. [114]

    Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence (UAI 2021) , pages =

    Variational Refinement for Importance Sampling Using the Forward Kullback–Leibler Divergence , author =. Proceedings of the Thirty-Seventh Conference on Uncertainty in Artificial Intelligence (UAI 2021) , pages =. 2021 , publisher =

    2021

  107. [115]

    , title =

    Neal, Radford M. , title =. Markov Chain Monte Carlo in Practice , editor =

  108. [116]

    MacKay, David J. C. , title =

  109. [117]

    and Sahani, Maneesh , title =

    Turner, Richard E. and Sahani, Maneesh , title =. Bayesian Time Series Models , editor =

  110. [118]

    tfp.experimental.vi.build\_factored\_surrogate\_posterior — TensorFlow Probability , author =

  111. [119]

    Variational API Quickstart — PyMC Examples , author =

  112. [120]

    TensorFlow Probability , author =

  113. [121]

    2024 , month =

    CmdStanPy Variational - How to get density parameters? , author =. 2024 , month =

    2024

  114. [122]

    Journal of Machine Learning Research , year =

    John Duchi and Elad Hazan and Yoram Singer , title =. Journal of Machine Learning Research , year =

  115. [123]

    International Conference on Learning Representations (ICLR) , year=

    Adam: A method for stochastic optimization , author=. International Conference on Learning Representations (ICLR) , year=

  116. [124]

    International Conference on Learning Representations (ICLR) , year=

    On the convergence of Adam and beyond , author=. International Conference on Learning Representations (ICLR) , year=

  117. [125]

    Advances in Neural Information Processing Systems (NeurIPS) , volume=

    The marginal value of adaptive gradient methods in machine learning , author=. Advances in Neural Information Processing Systems (NeurIPS) , volume=

  118. [126]

    Journal of Machine Learning Research , year =

    Ryan Giordano and Martin Ingram and Tamara Broderick , title =. Journal of Machine Learning Research , year =

  119. [127]

    Mason and Manuel Gomes and Richard Grieve and James R

    Alexina J. Mason and Manuel Gomes and Richard Grieve and James R. Carpenter , title =. Health Economics , year =

  120. [128]

    SAS/STAT \, 13.1 User’s Guide: The MCMC Procedure , publisher =

  121. [129]

    2022 , publisher =

    Kevin Davis Ross , title =. 2022 , publisher =

    2022

  122. [130]

    , title =

    Davidai, Shai and Gilovich, Thomas and Ross, Lee D. , title =. Proceedings of the National Academy of Sciences of the United States of America , year =

  123. [131]

    and Troxel, Andrea B

    Mehta, Shivan J. and Troxel, Andrea B. and Marcus, Noora and Jameson, Christina and Taylor, Devon and Asch, David A. and Volpp, Kevin G. , title =. JAMA Cardiology , year =

  124. [132]

    1998 , pages=

    Bayes Procedures , booktitle=. 1998 , pages=

    1998

  125. [133]

    Handbook of Uncertainty Quantification , publisher =

    Sampling via Measure Transport: An Introduction , author =. Handbook of Uncertainty Quantification , publisher =. 2016 , pages =

    2016

  126. [134]

    2024 , note =

    tfp.vi.fit\_surrogate\_posterior — TensorFlow Probability API Documentation , howpublished =. 2024 , note =

    2024

  127. [135]

    2025 , note =

    Bayesian Modeling with Joint Distribution , howpublished =. 2025 , note =

    2025

  128. [136]

    2024 , note =

    Variational Inference on Probabilistic Graphical Models with Joint Distributions , howpublished =. 2024 , note =

    2024

  129. [137]

    2025 , note =

    Fitting Generalized Linear Mixed-Effects Models Using Variational Inference , howpublished =. 2025 , note =

    2025

  130. [138]

    University of California Publications in Statistics , year =

    Le Cam, Lucien , title =. University of California Publications in Statistics , year =

  131. [139]

    2025 , note =

    Juan Camilo Orduz , title =. 2025 , note =

    2025

  132. [140]

    The Annals of Statistics , year =

    Nicolas Chopin , title =. The Annals of Statistics , year =

  133. [141]

    and Ott, Miles Q

    Johnson, Alicia A. and Ott, Miles Q. and Dogucu, Mine , doi =. Bayes Rules!

  134. [142]

    Detecting Urban

    Perry, Ni. Detecting Urban. npj Clean Air , volume =

  135. [143]

    Bayesian

    Lickley, Megan and Fletcher, Sarah , year = 2024, month = feb, journal =. Bayesian

    2024

  136. [144]

    International Conference on Learning Representations , year =

    Adaptive Gradient Methods with Dynamic Bound of Learning Rate , author =. International Conference on Learning Representations , year =

  137. [145]

    arXiv preprint arXiv:2410.19706 , year =

    Super Gradient Descent: Global Optimization requires Global Gradient , author =. arXiv preprint arXiv:2410.19706 , year =

    arXiv

  138. [146]

    2022 , url =

    Liao, Xuanzhi and Sahran, Shahnorbanun and Abdullah, Azizi and Shukor, Syaimak Abdul , journal =. 2022 , url =

    2022

  139. [147]

    An Adaptive Moment Estimation Method for Online

    Liu, Xin and Pan, Zhisong and Yang, Haimin and Zhou, Xingyu and Bai, Wei and Niu, Xianghua , journal =. An Adaptive Moment Estimation Method for Online. 2019 , url =

    2019

  140. [148]

    A Visual Explanation of Gradient Descent Methods: Momentum,

    Jiang, Lili , year =. A Visual Explanation of Gradient Descent Methods: Momentum,

  141. [149]

    Journal of the Royal Statistical Society Series C: Applied Statistics , volume=

    Bayesian state-space models for the modelling and prediction of the results of English Premier League football , author=. Journal of the Royal Statistical Society Series C: Applied Statistics , volume=. 2025 , publisher=

    2025

  142. [150]

    Journal of the Royal Statistical Society Series C: Applied Statistics , year =

    Species Sensitivity Distribution revisited: a Bayesian nonparametric approach , author =. Journal of the Royal Statistical Society Series C: Applied Statistics , year =

  143. [151]

    Journal of Open Source Software , volume =

    Kim, Eunseop , title =. Journal of Open Source Software , volume =

  144. [152]

    and Kuhl, Ellen , title =

    Linka, Kevin and Holzapfel, Gerhard A. and Kuhl, Ellen , title =. Computer Methods in Applied Mechanics and Engineering , volume =

  145. [153]

    Predicting Change: Approximate Inference Under Explicit Representation of Temporal Structure in Changing Environments , journal =

    Markovi. Predicting Change: Approximate Inference Under Explicit Representation of Temporal Structure in Changing Environments , journal =

  146. [154]

    Journal of Open Source Software , volume =

    Mahmood, Abuzar , title =. Journal of Open Source Software , volume =

  147. [155]

    Negative Emissions to Mitigate Earth System Risks , journal =

    Gasser, Thomas and Rezai, Armon and Cheritel, C. Negative Emissions to Mitigate Earth System Risks , journal =

  148. [156]

    Heliyon , volume =

    Hassan, Masoud Muhammed and Ismail, Halbast Rashid , title =. Heliyon , volume =

  149. [157]

    2026 , eprint=

    Large-scale empirical tuning and comparison of default optimizers for variational inference , author=. 2026 , eprint=

    2026

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.