Bayesian workflow diagnostics outperform unit tests for detecting and repairing statistically misspecified LLM-generated probabilistic programs across benchmarks and real generation tasks.
hub
Validating bayesian inference algorithms with simulation-based calibration
34 Pith papers cite this work, alongside 226 external citations. Polarity classification is still indexing.
34
Pith papers citing it
226
external citations · Pith
abstract
Verifying the correctness of Bayesian computation is challenging. This is especially true for complex models that are common in practice, as these require sophisticated model implementations and algorithms. In this paper we introduce \emph{simulation-based calibration} (SBC), a general procedure for validating inferences from Bayesian algorithms capable of generating posterior samples. This procedure not only identifies inaccurate computation and inconsistencies in model implementations but also provides graphical summaries that can indicate the nature of the problems that arise. We argue that SBC is a critical part of a robust Bayesian workflow, as well as being a useful tool for those developing computational algorithms and statistical software.
hub tools
citation-role summary
background 2
method 1
citation-polarity summary
representative citing papers
GraphNPE recovers a significantly lower central density for Boötes I consistent with a core while Draco remains marginally cuspy, and demonstrates that higher-order velocity moments reduce bias in dynamical modeling.
Introduces target-wise model-adequacy tests on Euclidean-admissible mock correlators to evaluate coverage properties of reported spectral uncertainties in lattice QCD reconstructions.
Extends NPE to mixed discrete-continuous parameter spaces via a factorized inference network combining an autoregressive classifier and generative model, trained jointly to yield accurate calibrated posteriors.
Simulation-based priors that depend on cluster morphology allow mock X-ray/SZ cluster samples to recover the input Hubble constant with 4% (100 clusters) and 1.5% (1000 clusters) precision plus a systematic floor of 0.6–0.8 km/s/Mpc.
Posterior Overlap 2.0 finds no lensed BBH pairs in O1–O4a (p_L < 0.6% for all pairs) and sets a 90% upper bound of 1.4% on the strong-lensing fraction.
SBI framework with GNN-on-sets and masked autoregressive flow recovers input cosmologies from eRASS1 mocks at 11.5% precision on Ω_m and 4.4% on σ_8 using 3259 clusters.
LLM embeddings condition a generative transformer to enable faster convergence, better performance, and generalization to unseen LHC processes using a single model.
Predictively consistent priors let complex Bayesian models match or beat the out-of-sample performance of selected simpler models across linear, logistic, and nonlinear examples without explicit selection.
smoothbp is an R package implementing fast Bayesian hierarchical piecewise regression with logistic smoothing, random effects, and spike-and-slab breakpoint selection via a custom Metropolis-within-Gibbs sampler in Rust.
Rest-frame optical galaxy SEDs from a 16-parameter SPS model are captured by five disentangled VAE latents (mass, young stars, dust, soft/hard ionization); metallicity and age are not independent drivers.
A neural spline-flow posterior estimator with a factorized spatio-temporal encoder recovers mass-loss rates and orbital parameters of colliding-wind binaries from 10-frame synthetic Hα photon-count time series.
Generative models for cosmological field-level inference can reproduce posterior means and cross-correlations yet fail to capture correct uncertainty geometry when validated against HMC reference samples.
GenSBI delivers JAX-native implementations of generative SBI methods with transformer backbones and reports near-ideal calibration scores on standard benchmarks.
Pointwise metrics compress marginal spectra in multimodal inverse problems, and a three-part protocol using CRPS, spectrum fidelity, and calibration reverses model rankings on synthetic and particle-physics benchmarks.
Bayesian joint model infers infectious virus shedding trajectories and derived infectiousness metrics from PCR and other proxies in SARS-CoV-2 using data from five cohorts of roughly 2000 infections.
SPIN performs bidirectional domain transfer in SBI to retain parameter mutual information from unlabeled real observations, improving real-world posterior inference under increasing misspecification.
Embedding selection mechanisms into generative simulators enables amortized Bayesian inference to produce debiased, well-calibrated posteriors without tractable likelihoods.
Simulation-based inference on Big Sobol Sequence halos at z=0.5 shows CMD+MFs improves σ8 and Ωm precision by ~27% over MFs alone and outperforms PS by ~45% in mass-selected samples at matched scales.
Score-based diffusion models learn the empirical distribution of real LIGO noise to enable unbiased gravitational-wave parameter estimation under only an additivity assumption.
New smooth invertible parameterization of anisotropic GF correlation length and diffusion matrix, with PC priors that penalize finite range and nonzero anisotropy for constant-parameter models.
Galaxy size-mass relations exhibit double power-law breaks at different pivot masses for quiescent versus bulge-dominated samples, coinciding with AGN activity scales.
In a controlled binary-black-hole benchmark, soft learned artifact-aware interval rescaling (LAIR) reduces marginal calibration error for frequency masks from 0.1195 to 0.0672 but is not uniformly better than raw intervals and is positioned as a diagnostic rather than a replacement for full posterio
Using ray-tracing simulations and simulation-based inference, the authors construct an AGN population that reproduces the cosmic X-ray background, number counts, and absorption properties, deriving an intrinsic Compton-thick fraction of 40±3%.
citing papers explorer
-
Calibration, Not Compilation: Detecting and Repairing Misspecified Probabilistic Programs Written by Language Models
Bayesian workflow diagnostics outperform unit tests for detecting and repairing statistically misspecified LLM-generated probabilistic programs across benchmarks and real generation tasks.
-
Dark Matter in Draco and Bo\"otes I: Hints of a Core in an Ultra-Faint Dwarf from Simulation-Based Inference
GraphNPE recovers a significantly lower central density for Boötes I consistent with a core while Draco remains marginally cuspy, and demonstrates that higher-order velocity moments reduce bias in dynamical modeling.
-
Conditional Model-Adequacy Tests for Spectral Uncertainty Claims in Lattice QCD
Introduces target-wise model-adequacy tests on Euclidean-admissible mock correlators to evaluate coverage properties of reported spectral uncertainties in lattice QCD reconstructions.
-
Mixed neural posterior estimation for simulators with discrete and continuous parameters
Extends NPE to mixed discrete-continuous parameter spaces via a factorized inference network combining an autoregressive classifier and generative model, trained jointly to yield accurate calibrated posteriors.
-
The Three Hundred Project: Validating $H_0$ inference from mock X-ray and millimetre analyses of galaxy clusters
Simulation-based priors that depend on cluster morphology allow mock X-ray/SZ cluster samples to recover the input Hubble constant with 4% (100 clusters) and 1.5% (1000 clusters) precision plus a systematic floor of 0.6–0.8 km/s/Mpc.
-
Search for strong lensing of gravitational waves in the binary black hole events from O1-O4a
Posterior Overlap 2.0 finds no lensed BBH pairs in O1–O4a (p_L < 0.6% for all pairs) and sets a 90% upper bound of 1.4% on the strong-lensing fraction.
-
Simulation-Based Inference for Cluster Cosmology with Set-Based Neural Network Architectures
SBI framework with GNN-on-sets and masked autoregressive flow recovers input cosmologies from eRASS1 mocks at 11.5% precision on Ω_m and 4.4% on σ_8 using 3259 clusters.
-
One Generator, Any Process: LLM-Conditioning for the LHC
LLM embeddings condition a generative transformer to enable faster convergence, better performance, and generalization to unseen LHC processes using a single model.
-
To select or not to select: predictively consistent priors instead of model selection
Predictively consistent priors let complex Bayesian models match or beat the out-of-sample performance of selected simpler models across linear, logistic, and nonlinear examples without explicit selection.
-
smoothbp: Fast Bayesian Hierarchical Piecewise Regression with Smoothed Transitions and Spike-and-Slab Model Selection
smoothbp is an R package implementing fast Bayesian hierarchical piecewise regression with logistic smoothing, random effects, and spike-and-slab breakpoint selection via a custom Metropolis-within-Gibbs sampler in Rust.
-
pop-cosmos: Disentangling galaxy properties from observables using data-driven approaches
Rest-frame optical galaxy SEDs from a 16-parameter SPS model are captured by five disentangled VAE latents (mass, young stars, dust, soft/hard ionization); metallicity and age are not independent drivers.
-
Amortized Simulation-Based Inference of Colliding-Wind Binaries from Short, Noisy Image Time Series
A neural spline-flow posterior estimator with a factorized spatio-temporal encoder recovers mass-loss rates and orbital parameters of colliding-wind binaries from 10-frame synthetic Hα photon-count time series.
-
Learning the Universe: Posterior Reliability of Neural Generative Models in High-Dimensional Field-Level Inference of Cosmic Initial Conditions
Generative models for cosmological field-level inference can reproduce posterior means and cross-correlations yet fail to capture correct uncertainty geometry when validated against HMC reference samples.
-
GenSBI: Generative Methods for Simulation-Based Inference in JAX
GenSBI delivers JAX-native implementations of generative SBI methods with transformer backbones and reports near-ideal calibration scores on standard benchmarks.
-
Pointwise Metrics Mislead: An Evaluation Protocol for Multimodal Inverse Problems
Pointwise metrics compress marginal spectra in multimodal inverse problems, and a three-part protocol using CRPS, spectrum fidelity, and calibration reverses model rankings on synthetic and particle-physics benchmarks.
-
Inferring infectiousness: a joint model of the within-host viral kinetics of SARS-CoV-2
Bayesian joint model infers infectious virus shedding trajectories and derived infectiousness metrics from PCR and other proxies in SARS-CoV-2 using data from five cohorts of roughly 2000 infections.
-
Information-Preserving Domain Transfer with Unlabeled Data in Misspecified Simulation-Based Inference
SPIN performs bidirectional domain transfer in SBI to retain parameter mutual information from unlabeled real observations, improving real-world posterior inference under increasing misspecification.
-
Overcoming Selection Bias in Statistical Studies With Amortized Bayesian Inference
Embedding selection mechanisms into generative simulators enables amortized Bayesian inference to produce debiased, well-calibrated posteriors without tractable likelihoods.
-
Quantifying Weighted Morphological Content of Large-Scale Structures via Simulation-Based Inference
Simulation-based inference on Big Sobol Sequence halos at z=0.5 shows CMD+MFs improves σ8 and Ωm precision by ~27% over MFs alone and outperforms PS by ~45% in mass-selected samples at matched scales.
-
Gravitational-Wave Parameter Estimation in non-Gaussian noise using Score-Based Likelihood Characterization
Score-based diffusion models learn the empirical distribution of real LIGO noise to enable unbiased gravitational-wave parameter estimation under only an additivity assumption.
-
A parameterization of anisotropic Gaussian fields with penalized complexity priors
New smooth invertible parameterization of anisotropic GF correlation length and diffusion matrix, with PC priors that penalize finite range and nonzero anisotropy for constant-parameter models.
-
pop-cosmos: Galaxy size evolution across structural and star-formation classifications in COSMOS-Web
Galaxy size-mass relations exhibit double power-law breaks at different pivot masses for quiescent versus bulge-dominated samples, coinciding with AGN activity scales.
-
Artifact-Conditioned Interval Diagnostics for Flow-Matching Neural Posterior Estimation in a Controlled Gravitational-Wave Benchmark
In a controlled binary-black-hole benchmark, soft learned artifact-aware interval rescaling (LAIR) reduces marginal calibration error for frequency masks from 0.1195 to 0.0672 but is not uniformly better than raw intervals and is positioned as a diagnostic rather than a replacement for full posterio
-
Population synthesis of active galactic nuclei based on the radiation-regulated unification model
Using ray-tracing simulations and simulation-based inference, the authors construct an AGN population that reproduces the cosmic X-ray background, number counts, and absorption properties, deriving an intrinsic Compton-thick fraction of 40±3%.
-
Separating Intrinsic Ambiguity from Estimation Uncertainty in Deep Generative Models for Linear Inverse Problems
Introduces a cascade-based structural decomposition of posterior uncertainty to isolate intrinsic ambiguity from estimation uncertainty in deep generative models for linear inverse problems.
-
Neural Posterior Estimation of Terrain Parameters from Radar Sounder Data
Neural posterior estimation trained on GPU-simulated radar data enables calibrated probabilistic inversion of terrain parameters and transfers to real Mars radar profiles.
-
MIRA: A Score for Conditional Distribution Accuracy and Model Comparison
MIRA is a new analytic score for conditional distribution accuracy derived from equal probability mass assignment, enabling Bayesian model comparison via direct posterior validation.
-
Tests of General Relativity with Binary Black Holes from the second LIGO-Virgo Gravitational-Wave Transient Catalog
No evidence for deviations from general relativity is found in LIGO-Virgo binary black hole events, with improved constraints on waveform parameters, graviton mass, and ringdown properties.
-
Neutron Star Mass across Binary Pulsar Subpopulations: Mass-Spin Correlation, Mass Distributions, and Moment of Inertia Effects
Hierarchical Bayesian analysis of ~50 binary pulsars finds moderate evidence for mass-spin anti-correlation (ρ = -0.26) in the recycled population, plus a small mass offset by white-dwarf companion type and confirmation of companion-mass–eccentricity correlation in DNS systems.
-
Bayesian Extreme Value Theory with Hawkes-AR-Gumbel Dependence for Extreme CVaR Estimation in Operational Risk
Bayesian EVT with Hawkes-AR-Gumbel dependence estimates CVaR up to 99.995% on simulated operational risk data and outperforms independent and shared-factor baselines.
-
Bayesian inference for compact binary coalescences with BILBY: Validation and application to the first LIGO--Virgo gravitational-wave transient catalogue
BILBY is validated on simulated compact binary signals and reproduces the eleven GWTC-1 results with configuration and output files provided for reproduction.
-
Machine Learning Techniques for Astrophysics and Cosmology: Simulation-Based Inference
Simulation-based inference uses neural networks trained on simulations to enable parameter inference in cosmology and astrophysics where traditional likelihood calculations are intractable.
-
Application of Machine Learning to 21 cm Cosmology
Review chapter organizing machine learning methods for 21 cm cosmology into observation, theory, and inference domains.
-
A Review of Diffusion-based Simulation-Based Inference: Foundations and Applications in Non-Ideal Data Scenarios
A synthesis of diffusion-based simulation-based inference methods that address model misspecification, irregular observations, and missing data in scientific applications.