pith. sign in

arxiv: 2605.23791 · v1 · pith:BFIBKZKHnew · submitted 2026-05-22 · 📊 stat.ME

Joint Bayesian models for validating spatial health-event databases against a gold standard: separating global and local discrepancies

Mathias Brugel , Florine Kempf , Camille Ternynck , Marta Blangiardo , Micha\"el G\'enin This is my paper

Pith reviewed 2026-05-25 03:12 UTC · model grok-4.3

classification 📊 stat.ME
keywords spatial database validationBayesian hierarchical modelsdisease mappingdata reuseshared component modelglobal discrepancieslocal discrepancieshealth-event databases
0
0 comments X

The pith

Bayesian models separate global and local discrepancies when validating spatial health databases against a gold standard

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper proposes a Bayesian framework to validate a candidate-for-reuse database against a gold standard by modeling the candidate as a departure from the standard using random and structured error models. Global disagreement is measured by the intercept difference RR_global while local disagreement uses exceedance probabilities of the database-specific error term. These are compared against a shared component model and tested on simulated perturbations including null, uniform, clustered, and random cases plus an application to Crohn's disease data. REM and SEM proved sensitive and specific to local issues while RR_global recovered map-wide shifts in all cases, and all models found the candidate reproduced spatial structures with an overall signal about 7 percent lower. A sympathetic reader would care because the method supplies a concrete tool for assessing whether medico-administrative spatial data can be reused without hidden systematic distortions.

Core claim

The authors establish that their Bayesian error-model family accurately recovers global map-wide shifts via RR_global across all models and perturbation scenarios, that REM and SEM are both sensitive and specific to local discrepancies while the shared component model is more conservative, and that in the EPIMAD Crohn's disease application all models agree the candidate database reproduces global and local spatial structures with an overall signal about 7 percent lower.

What carries the argument

The error-model family in which the candidate database is modeled as a departure from the gold standard, using database-specific intercept difference RR_global for global disagreement and exceedance probability of the database-specific error term for local disagreement, compared against a shared component model.

If this is right

  • RR_global accurately recovered map-wide shifts across all models and scenarios.
  • REM and SEM were both sensitive and specific to local discrepancies.
  • SCM was more conservative in detecting local discrepancies.
  • In the Crohn's disease application all models concluded the candidate reproduced global and local spatial structures with an overall signal about 7 percent lower.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same validation structure could be applied to other administrative health databases to quantify how much reuse distorts incidence maps in different regions.
  • Extending the framework to count outcomes with different distributions would test whether the global-local separation remains stable when the underlying likelihood changes.
  • Using the method on paired databases with known external validation sources would provide a direct check on whether the reported 7 percent signal reduction matches independent measurements.

Load-bearing premise

The gold standard database is treated as error-free truth and the chosen random or structured error models are assumed to correctly capture the true form of discrepancies without misspecification biasing the global or local estimates.

What would settle it

A simulation study in which the true discrepancies follow a structure outside the assumed random and structured families, followed by checking whether RR_global still recovers the known map-wide shift and whether the sensitivity-specificity performance for local detection holds.

Figures

Figures reproduced from arXiv: 2605.23791 by Camille Ternynck, Florine Kempf, Marta Blangiardo, Mathias Brugel, Micha\"el G\'enin.

Figure 1
Figure 1. Figure 1: Posterior median of the global relative-risk contrast, 𝑅𝑅global = exp(Δ), across simulation scenarios and perturbation settings, for the random error model (REM), shared component model (SCM), and structured error model (SEM), together with corresponding 95% credible intervals (CrIs). The dashed horizontal line indicates 𝑅𝑅global = 1, corresponding to the absence of global discrepancy between the two datab… view at source ↗
Figure 2
Figure 2. Figure 2: Mean local classification performance across simulation settings for clustered perturbations (S3) and random perturbations (S4), summarised by sensitivity, specificity, and false discovery rate. Results are shown for the REM, SCM, and SEM under the null-referenced exceedance probability (NREP) and the robustly centred exceedance probability (RCEP), with several decision thresholds. Darker shading indicates… view at source ↗
Figure 3
Figure 3. Figure 3: Mean Matthews correlation coefficient (MCC) across simulation settings for clustered perturbations (S3) and random perturbations (S4), for the REM, SCM, and SEM under the null-referenced exceedance probability (NREP) and the robustly centred exceedance probability (RCEP). Higher values indicate better overall classification performance by jointly accounting for true and false positives and negatives. This … view at source ↗
Figure 4
Figure 4. Figure 4: Smoothed area-level relative risks for Crohn’s disease obtained separately from the EPIMAD registry (A) and the French national hospital discharge database (PMSI; B) using source-specific Poisson models with BYM2 spatial effects. In the EPIMAD source, the mapped signal is based on cumulative incidence over 1988–2014, whereas in PMSI it is based on hospital-derived counts over 2007–2014. A common colour sca… view at source ↗
read the original abstract

The reuse of medico-administrative and synthetic spatial data may overcome some limitations of population-based registries, provided rigorous validation is performed. However, no tool exists to spatially validate a candidate-for-reuse database (CFRD) against a gold standard (GS). We propose a Bayesian framework for two-dimensional (global and local) map-to-map validation of spatial health-event databases. We consider an error-model family (random [REM] and structured [SEM]) in which the CFRD is modelled as a departure from the GS. Both are compared with a shared component model (SCM). Global disagreement is assessed using the database-specific intercept difference ($RR_{\mathrm{global}}$), while local disagreement is measured by the exceedance probability of the database-specific error term. Disturbance scenarios included null, uniform, clustered, and random perturbations in the CFRD. Sensitivity, specificity, false detection rate, and Matthews Correlation Coefficient assessed detection performance. $RR_{\mathrm{global}}$ accurately recovered map-wide shifts across all models and scenarios. REM and SEM behaved were both sensitive and specific to local discrepancies. SCM was more conservative. Applied to Crohn's disease data from the EPIMAD registry and a CFRD, all models reached the same conclusion: the CFRD reproduced global and local spatial structures with an overall signal about 7\% lower. Extensions to other outcome distributions, spatio-temporal models and calibration constitute natural next steps. \textit{Keywords:} data reuse; spatial database validation; Bayesian hierarchical models; disease mapping; shared component model.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 3 minor

Summary. The paper proposes a Bayesian hierarchical modeling framework for two-dimensional (global and local) validation of a candidate-for-reuse database (CFRD) against a gold standard (GS) spatial health-event database. It introduces random (REM) and structured (SEM) error models in which the CFRD is modeled as a departure from the GS, compares them to a shared component model (SCM), and uses the database-specific intercept difference (RR_global) for global disagreement and exceedance probabilities for local disagreement. Performance is assessed via simulations under null, uniform, clustered, and random perturbation scenarios using sensitivity, specificity, false detection rate, and Matthews correlation coefficient. The method is applied to Crohn's disease incidence data from the EPIMAD registry and a CFRD, concluding that the CFRD reproduces global and local spatial structures with an overall signal approximately 7% lower.

Significance. If the central claims hold, the work fills a gap by providing the first dedicated spatial validation tool for reused medico-administrative and synthetic health databases against a gold standard. The simulation-based recovery of known global shifts by RR_global across models and the sensitivity/specificity results for REM/SEM (with SCM more conservative) offer direct evidence of utility. Model agreement on the ~7% global offset in the real-data application further supports practical value for data reuse in epidemiology and disease mapping.

major comments (2)
  1. [Abstract] The abstract claims that 'RR_global accurately recovered map-wide shifts across all models and scenarios' and that 'REM and SEM were both sensitive and specific,' yet provides no quantitative recovery metrics, error bars, or simulation sample sizes. If these performance statements are load-bearing for the central claim of reliable validation, the full methods and results sections must supply the corresponding tables or figures with numerical values to allow verification.
  2. [Abstract (and methods)] The framework treats the gold standard as error-free truth and assumes the REM/SEM error families correctly capture the true discrepancy structure. This assumption is load-bearing for both the simulation recovery claims and the real-data conclusion of a 7% lower signal; any misspecification could bias RR_global or local exceedance probabilities. A sensitivity analysis to alternative error structures or a discussion of robustness would strengthen the central claim.
minor comments (3)
  1. [Abstract] The sentence 'REM and SEM behaved were both sensitive and specific' contains a grammatical error that should be corrected.
  2. [Abstract / Introduction] The abstract states that 'no tool exists' for spatial validation; if this is intended as a novelty claim, it should be supported by a brief literature review in the introduction citing any related (even non-Bayesian) map-comparison methods.
  3. [Abstract] The keywords are appropriate, but the abstract could usefully include one or two key model equations (e.g., the form of RR_global) to make the contribution more self-contained.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive comments and positive recommendation. We address each major comment below.

read point-by-point responses

  1. Referee: [Abstract] The abstract claims that 'RR_global accurately recovered map-wide shifts across all models and scenarios' and that 'REM and SEM were both sensitive and specific,' yet provides no quantitative recovery metrics, error bars, or simulation sample sizes. If these performance statements are load-bearing for the central claim of reliable validation, the full methods and results sections must supply the corresponding tables or figures with numerical values to allow verification.

    Authors: We agree that the abstract would be strengthened by including key quantitative results. The full manuscript already reports these in Section 3 (simulation study), with tables providing sensitivity, specificity, FDR, MCC, and RR_global point estimates plus 95% credible intervals across 1000 replicates per scenario. We will revise the abstract to report representative numerical values (e.g., RR_global recovery ranges and average sensitivity/specificity). revision: yes

  2. Referee: [Abstract (and methods)] The framework treats the gold standard as error-free truth and assumes the REM/SEM error families correctly capture the true discrepancy structure. This assumption is load-bearing for both the simulation recovery claims and the real-data conclusion of a 7% lower signal; any misspecification could bias RR_global or local exceedance probabilities. A sensitivity analysis to alternative error structures or a discussion of robustness would strengthen the central claim.

    Authors: We acknowledge that the error-free GS assumption and the REM/SEM families are central modeling choices. The simulations recover known perturbations under these structures, and the real-data results are consistent across REM, SEM, and the more conservative SCM. We will add a dedicated paragraph in the Discussion addressing potential misspecification and the robustness gained from model comparison. A comprehensive sensitivity analysis to other error structures lies outside the present scope but is identified as future work. revision: partial

Circularity Check

0 steps flagged

No significant circularity identified

full rationale

The paper introduces Bayesian models (REM, SEM, SCM) for map-to-map validation of spatial databases, defines RR_global as the intercept difference and local exceedance probabilities for discrepancies, then evaluates these via simulations with known perturbations (null, uniform, clustered, random) and applies them to Crohn's disease data. No load-bearing step reduces a prediction or result to a fitted parameter by construction, nor does any central claim rest on self-citation chains or imported uniqueness theorems; the simulation recovery metrics are computed against independently generated ground-truth scenarios, and the real-data conclusion of ~7% global offset is a direct model output rather than a renaming or self-referential fit. The framework is self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review provides no explicit free parameters, axioms, or invented entities; RR_global is referenced as a derived quantity but its estimation details are absent.

pith-pipeline@v0.9.0 · 5827 in / 1187 out tokens · 43135 ms · 2026-05-25T03:12:25.765567+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

29 extracted references · 29 canonical work pages

  1. [1]

    Ahmed, M.-S., Cucala, L., and Genin, M. (2021). Spatial autoregressive models for scan statistic.Journal of Spatial Econometrics, 2(1):1–20

    work page 2021

  2. [2]

    Anselin, L. (1995). Local indicators of spatial association—LISA.Geographical Analysis, 27(2):93–115

    work page 1995

  3. [3]

    Besag, J., York, J., and Mollié, A. (1991). Bayesian image restoration, with two applications in spatial statistics. Annals of the Institute of Statistical Mathematics, 43(1):1–20

    work page 1991

  4. [4]

    and Cameletti, M

    Blangiardo, M. and Cameletti, M. (2015).Spatial and Spatio-Temporal Bayesian Models with R-INLA. John Wiley & Sons, Chichester, UK

    work page 2015

  5. [5]

    and Boyer, L

    Boussat, B. and Boyer, L. (2024). Embracing change and advancing public health: The new era of the journal of epidemiology and population health.Journal of Epidemiology and Population Health, 72(1):202383

    work page 2024

  6. [6]

    Cucala, L., Genin, M., Lanier, C., and Occelli, F. (2017). A multivariate gaussian scan statistic for spatial data. Spatial Statistics, 21:66–74

    work page 2017

  7. [7]

    Cucala, L., Genin, M., Occelli, F., and Soula, J. (2019). A multivariate nonparametric scan statistic for spatial data.Spatial Statistics, 29:1–14

    work page 2019

  8. [8]

    Etxeberria, J., Goicoa, T., and Ugarte, M. D. (2023). Using mortality to predict incidence for rare and lethal cancers in very small areas.Biometrical Journal, 65(3):e2200017

    work page 2023

  9. [9]

    Fuentes-Santos, I., González-Manteiga, W., and Mateu, J. (2017). A nonparametric test for the comparison of first-order structures of spatial point processes.Spatial Statistics, 22(Part 2):240–260

    work page 2017

  10. [10]

    Fuentes-Santos, I., González-Manteiga, W., and Mateu, J. (2023). Testing similarity between first-order intensities of spatial point processes: a comparative study.Spatial Statistics, 58:100816

    work page 2023

  11. [11]

    Vasseur, F., Cortot, A., Colombel, J.-F., and Gower-Rousseau, C. (2013). Space-time clusters of Crohn’s disease in northern France.Journal of Public Health, 21(6):497–504

    work page 2013

  12. [12]

    Malapel, M., Sarter, H., Gower-Rousseau, C., and Ficheur, G. (2020). Fine-scale geographical distribution andecologicalriskfactorsforcrohn’sdiseaseinfrance(2007–2014).AlimentaryPharmacology&Therapeutics, 51(1):139–148. Gómez-Rubio, V., Palmí-Perales, F., López-Abente, G., Ramis-Prieto, R., and Fernández-Navarro, P. (2019). Bayesian joint spatio-temporal a...

    work page 2020

  13. [13]

    Hahn, U. (2012). A studentized permutation test for the comparison of spatial point patterns.Journal of the American Statistical Association, 107(498):754–764

    work page 2012

  14. [14]

    and Best, N

    Knorr-Held, L. and Best, N. G. (2001). A shared component model for detecting joint and selective clustering of two diseases.Journal of the Royal Statistical Society: Series A (Statistics in Society), 164(1):73–85

    work page 2001

  15. [15]

    Kulldorff, M., Huang, L., and Konty, K. (2009). A scan statistic for continuous data based on the normal probability model.International Journal of Health Geographics, 8(1):58

    work page 2009

  16. [16]

    K., Kleinman, K., and Platt, R

    Kulldorff, M., Mostashari, F., Duczmal, L., Yih, W. K., Kleinman, K., and Platt, R. (2007). Multivariate scan statistics for disease surveillance.Statistics in Medicine, 26(8):1824–1833

    work page 2007

  17. [17]

    Lee, S.-I. (2001). Developing a bivariate spatial association measure: an integration of pearson’s r and moran’s i.Journal of Geographical Systems, 3(4):369–385

    work page 2001

  18. [18]

    G., Lei, X., and Breslow, N

    Leroux, B. G., Lei, X., and Breslow, N. (2000). Estimation of disease rates in small areas: a new mixed model for spatial dependence. In Halloran, M. E. and Berry, D., editors,Statistical Models in Epidemiology, the Environment, and Clinical Trials, pages 179–191, New York, NY. Springer

    work page 2000

  19. [19]

    S., Yorita, K

    Levine, R. S., Yorita, K. L., Walsh, M. C., and Reynolds, M. G. (2009). A method for statistically comparing spatial distribution maps.International Journal of Health Geographics, 8(1):7

    work page 2009

  20. [20]

    Lin, J. (2023). Comparison of moran’s i and geary’s c in multivariate spatial pattern analysis.Geographical Analysis, 55(4):685–702

    work page 2023

  21. [21]

    Geographical variability and environmental risk factors in inflammatory bowel disease.Gut, 62(4):630–649

    Colombel, J.-F., and Epidemiology and Natural History Task Force of the International Organization of Inflammatory Bowel Disease (IOIBD) (2013). Geographical variability and environmental risk factors in inflammatory bowel disease.Gut, 62(4):630–649

    work page 2013

  22. [22]

    Paiva, T., Chakraborty, A., Reiter, J., and Gelfand, A. (2014). Imputation of confidential data sets with spatial locations using disease mapping models.Statistics in Medicine, 33(11):1928–1945

    work page 2014

  23. [23]

    Quick, H. (2021). Generating poisson-distributed differentially private synthetic data.Journal of the Royal Statistical Society Series A: Statistics in Society, 184(3):1093–1108

    work page 2021

  24. [24]

    and Waller, L

    Quick, H. and Waller, L. A. (2018). Using spatiotemporal models to generate synthetic data for public use. Spatial and Spatio-temporal Epidemiology, 27:37–45

    work page 2018

  25. [25]

    M., Iwaz, J., Gomez, F., Olive, F., Polazzi, S., Schott, A

    Remontet, L., Mitton, N., Couris, C. M., Iwaz, J., Gomez, F., Olive, F., Polazzi, S., Schott, A. M., Trombert, B., Bossard,N.,etal.(2008). Isitpossibletoestimatetheincidenceofbreastcancerfrommedico-administrative databases?European journal of epidemiology, 23(10):681–688. Retegui,G.,Etxeberria,J.,andUgarte,M.D.(2021). EstimatingLOCPcancermortalityratesins...

    work page 2008

  26. [26]

    H., Simpson, D., and Rue, H

    Riebler, A., Sørbye, S. H., Simpson, D., and Rue, H. (2016). An intuitive bayesian spatial model for disease mapping that accounts for scaling.Statistical Methods in Medical Research, 25(4):1145–1165

    work page 2016

  27. [27]

    Rue, H., Martino, S., and Chopin, N. (2009). Approximate bayesian inference for latent gaussian models by using integrated nested laplace approximations.Journal of the Royal Statistical Society: Series B (Statistical Methodology), 71(2):319–392

    work page 2009

  28. [28]

    Rustamov, R. M. and Klosowski, J. T. (2020). Kernel mean embedding based hypothesis tests for comparing spatial point patterns.Spatial Statistics, 38:100459

    work page 2020

  29. [29]

    L., Wadmann, S., and Hoeyer, K

    Skovgaard, L. L., Wadmann, S., and Hoeyer, K. (2019). A review of attitudes towards the reuse of health data among people in the european union: the primacy of purpose and the common good.Health Policy, 123(6):564–571. The Lancet (2025). Cancer registries: the bedrock of global cancer care.The Lancet, 405(10476):353. 18 S1 Supplementary materials For each...

    work page 2019