Validating Bayesian Inference Algorithms with Simulation-Based Calibration
Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel pith:CXCVQBN3record.jsonopen to challenge →
read the original abstract
Verifying the correctness of Bayesian computation is challenging. This is especially true for complex models that are common in practice, as these require sophisticated model implementations and algorithms. In this paper we introduce \emph{simulation-based calibration} (SBC), a general procedure for validating inferences from Bayesian algorithms capable of generating posterior samples. This procedure not only identifies inaccurate computation and inconsistencies in model implementations but also provides graphical summaries that can indicate the nature of the problems that arise. We argue that SBC is a critical part of a robust Bayesian workflow, as well as being a useful tool for those developing computational algorithms and statistical software.
This paper has not been read by Pith yet.
Forward citations
Cited by 38 Pith papers
-
Calibration, Not Compilation: Detecting and Repairing Misspecified Probabilistic Programs Written by Language Models
Bayesian workflow diagnostics outperform unit tests for detecting and repairing statistically misspecified LLM-generated probabilistic programs across benchmarks and real generation tasks.
-
Dark Matter in Draco and Bo\"otes I: Hints of a Core in an Ultra-Faint Dwarf from Simulation-Based Inference
GraphNPE recovers a significantly lower central density for Boötes I consistent with a core while Draco remains marginally cuspy, and demonstrates that higher-order velocity moments reduce bias in dynamical modeling.
-
One Generator, Any Process: LLM-Conditioning for the LHC
LLM embeddings condition generative networks for LHC events, yielding faster convergence, higher quality, and generalization to unseen processes.
-
Conditional Model-Adequacy Tests for Spectral Uncertainty Claims in Lattice QCD
Introduces target-wise model-adequacy tests on Euclidean-admissible mock correlators to evaluate coverage properties of reported spectral uncertainties in lattice QCD reconstructions.
-
Mixed neural posterior estimation for simulators with discrete and continuous parameters
Extends NPE to mixed discrete-continuous parameter spaces via a factorized inference network combining an autoregressive classifier and generative model, trained jointly to yield accurate calibrated posteriors.
-
Simulation-Based Inference for Cluster Cosmology with Set-Based Neural Network Architectures
SBI framework with GNN-on-sets and masked autoregressive flow recovers input cosmologies from eRASS1 mocks at 11.5% precision on Ω_m and 4.4% on σ_8 using 3259 clusters.
-
One Generator, Any Process: LLM-Conditioning for the LHC
LLM embeddings condition a generative transformer to enable faster convergence, better performance, and generalization to unseen LHC processes using a single model.
-
To select or not to select: predictively consistent priors instead of model selection
Predictively consistent priors let complex Bayesian models match or beat the out-of-sample performance of selected simpler models across linear, logistic, and nonlinear examples without explicit selection.
-
smoothbp: Fast Bayesian Hierarchical Piecewise Regression with Smoothed Transitions and Spike-and-Slab Model Selection
smoothbp is an R package implementing fast Bayesian hierarchical piecewise regression with logistic smoothing, random effects, and spike-and-slab breakpoint selection via a custom Metropolis-within-Gibbs sampler in Rust.
-
pop-cosmos: Disentangling galaxy properties from observables using data-driven approaches
A beta-VAE analysis of pop-cosmos models finds that five latent dimensions capture the rest-frame optical SED, corresponding to stellar mass, recent star formation, dust, and two gas ionization states.
-
Amortized Simulation-Based Inference of Colliding-Wind Binaries from Short, Noisy Image Time Series
Amortized SBI with spatio-temporal embeddings infers seven CWB parameters from 10-frame Hα time series with well-calibrated posteriors on synthetic data.
-
Learning the Universe: Posterior Reliability of Neural Generative Models in High-Dimensional Field-Level Inference of Cosmic Initial Conditions
Generative models for cosmological field-level inference can reproduce posterior means and cross-correlations yet fail to capture correct uncertainty geometry when validated against HMC reference samples.
-
GenSBI: Generative Methods for Simulation-Based Inference in JAX
GenSBI delivers JAX-native implementations of generative SBI methods with transformer backbones and reports near-ideal calibration scores on standard benchmarks.
-
Pointwise Metrics Mislead: An Evaluation Protocol for Multimodal Inverse Problems
Pointwise metrics compress marginal spectra in multimodal inverse problems, and a three-part protocol using CRPS, spectrum fidelity, and calibration reverses model rankings on synthetic and particle-physics benchmarks.
-
Inferring infectiousness: a joint model of the within-host viral kinetics of SARS-CoV-2
Bayesian joint model infers infectious virus shedding trajectories and derived infectiousness metrics from PCR and other proxies in SARS-CoV-2 using data from five cohorts of roughly 2000 infections.
-
Information-Preserving Domain Transfer with Unlabeled Data in Misspecified Simulation-Based Inference
SPIN performs bidirectional domain transfer in SBI to retain parameter mutual information from unlabeled real observations, improving real-world posterior inference under increasing misspecification.
-
Information-Preserving Domain Transfer with Unlabeled Data in Misspecified Simulation-Based Inference
SPIN improves posterior inference under model misspecification in SBI by learning parameter-relevant information-preserving domain transfers from unpaired unlabeled real-world data.
-
Overcoming Selection Bias in Statistical Studies With Amortized Bayesian Inference
Embedding selection mechanisms into generative simulators enables amortized Bayesian inference to produce debiased, well-calibrated posteriors without tractable likelihoods.
-
Quantifying Weighted Morphological Content of Large-Scale Structures via Simulation-Based Inference
Simulation-based inference on Big Sobol Sequence halos at z=0.5 shows CMD+MFs improves σ8 and Ωm precision by ~27% over MFs alone and outperforms PS by ~45% in mass-selected samples at matched scales.
-
Gravitational-Wave Parameter Estimation in non-Gaussian noise using Score-Based Likelihood Characterization
Score-based diffusion models learn the empirical distribution of real LIGO noise to enable unbiased gravitational-wave parameter estimation under only an additivity assumption.
-
A parameterization of anisotropic Gaussian fields with penalized complexity priors
New smooth invertible parameterization of anisotropic GF correlation length and diffusion matrix, with PC priors that penalize finite range and nonzero anisotropy for constant-parameter models.
-
pop-cosmos: Galaxy size evolution across structural and star-formation classifications in COSMOS-Web
Galaxy size-mass relations exhibit double power-law breaks at different pivot masses for quiescent versus bulge-dominated samples, coinciding with AGN activity scales.
-
Artifact-Conditioned Interval Diagnostics for Flow-Matching Neural Posterior Estimation in a Controlled Gravitational-Wave Benchmark
In a controlled binary-black-hole benchmark, soft learned artifact-aware interval rescaling (LAIR) reduces marginal calibration error for frequency masks from 0.1195 to 0.0672 but is not uniformly better than raw inte...
-
Population synthesis of active galactic nuclei based on the radiation-regulated unification model
Using ray-tracing simulations and simulation-based inference, the authors construct an AGN population that reproduces the cosmic X-ray background, number counts, and absorption properties, deriving an intrinsic Compto...
-
Separating Intrinsic Ambiguity from Estimation Uncertainty in Deep Generative Models for Linear Inverse Problems
Introduces a cascade-based structural decomposition of posterior uncertainty to isolate intrinsic ambiguity from estimation uncertainty in deep generative models for linear inverse problems.
-
Neural Posterior Estimation of Terrain Parameters from Radar Sounder Data
Neural posterior estimation trained on GPU-simulated radar data enables calibrated probabilistic inversion of terrain parameters and transfers to real Mars radar profiles.
-
Neural Posterior Estimation of Terrain Parameters from Radar Sounder Data
Neural posterior estimation trained on simulated radar data enables probabilistic inference of terrain parameters from real Mars radar sounder profiles while conditioning on reference surface assumptions.
-
MIRA: A Score for Conditional Distribution Accuracy and Model Comparison
MIRA is a new analytic score for conditional distribution accuracy derived from equal probability mass assignment, enabling Bayesian model comparison via direct posterior validation.
-
Tests of General Relativity with Binary Black Holes from the second LIGO-Virgo Gravitational-Wave Transient Catalog
No evidence for deviations from general relativity is found in LIGO-Virgo binary black hole events, with improved constraints on waveform parameters, graviton mass, and ringdown properties.
-
Neutron Star Mass across Binary Pulsar Subpopulations: Mass-Spin Correlation, Mass Distributions, and Moment of Inertia Effects
Hierarchical Bayesian analysis of ~50 binary pulsars finds moderate evidence for mass-spin anti-correlation (ρ = -0.26) in the recycled population, plus a small mass offset by white-dwarf companion type and confirmati...
-
Bayesian Extreme Value Theory with Hawkes-AR-Gumbel Dependence for Extreme CVaR Estimation in Operational Risk
Bayesian EVT with Hawkes-AR-Gumbel dependence estimates CVaR up to 99.995% on simulated operational risk data and outperforms independent and shared-factor baselines.
-
Bayesian inference for compact binary coalescences with BILBY: Validation and application to the first LIGO--Virgo gravitational-wave transient catalogue
BILBY is validated on simulated compact binary signals and reproduces the eleven GWTC-1 results with configuration and output files provided for reproduction.
-
Machine Learning Techniques for Astrophysics and Cosmology: Simulation-Based Inference
Simulation-based inference uses neural networks trained on simulations to enable parameter inference in cosmology and astrophysics where traditional likelihood calculations are intractable.
-
Application of Machine Learning to 21 cm Cosmology
Machine learning helps 21 cm cosmology most when it preserves physically relevant structure and propagates uncertainty explicitly instead of replacing the forward model.
-
Application of Machine Learning to 21 cm Cosmology
Review chapter organizing machine learning methods for 21 cm cosmology into observation, theory, and inference domains.
-
Application of Machine Learning to 21 cm Cosmology
A review chapter groups machine learning methods for 21 cm cosmology by their pipeline roles in handling contaminated data, accelerating simulations, and inferring astrophysical parameters.
-
A Review of Diffusion-based Simulation-Based Inference: Foundations and Applications in Non-Ideal Data Scenarios
A synthesis of diffusion-based simulation-based inference methods that address model misspecification, irregular observations, and missing data in scientific applications.
-
Application of Machine Learning to 21 cm Cosmology
Machine learning can address data contamination, accelerate modeling, and aid inference in 21 cm cosmology when it preserves physical structure and uncertainty rather than acting as an opaque replacement.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.