A Bayesian quantification of consistency in correlated datasets

We present three tiers of Bayesian consistency tests for the general case of $correlated$ datasets. Building on duplicates of the model parameters assigned to each dataset, these tests range from Bayesian evidence ratios as a global summary statistic, to posterior distributions of model parameter differences, to consistency tests in the data domain derived from posterior predictive distributions. For each test we motivate meaningful threshold criteria for the internal consistency of datasets. Without loss of generality we focus on mutually exclusive, correlated subsets of the same dataset in this work. As an application, we revisit the consistency analysis of the two-point weak lensing shear correlation functions measured from KiDS-450 data. We split this dataset according to large vs. small angular scales, tomographic redshift bin combinations, and estimator type. We do not find any evidence for significant internal tension in the KiDS-450 data, with significances below $3\, \sigma$ in all cases. Software and data used in this analysis can be found at http://kids.strw.leidenuniv.nl/sciencedata.php