RECORD
Statistical coverage for supersymmetric parameter estimation: a case study with direct detection of dark matter
High Energy Physics - Phenomenology · Submitted 18 Nov 2010
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim We observe both under- and over-coverage, which in some cases vary quite dramatically when benchmarks or priors are modified.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Models of weak-scale supersymmetry offer viable dark matter (DM) candidates. Their parameter spaces are however rather large and complex, such that pinning down the actual parameter values from experimental data can depend strongly on the employed statistical framework and scanning algorithm. In frequentist parameter estimation, a central requirement for properly constructed confidence intervals is that they cover true parameter values, preferably at exactly the stated confidence level when experiments are repeated infinitely many times. Since most widely-used scanning techniques are optimised for Bayesian statistics, one needs to assess their abilities in providing correct confidence intervals in terms of the statistical coverage. Here we investigate this for the Constrained Minimal Supersymmetric Standard Model (CMSSM) when only constrained by data from direct searches for dark matter. We construct confidence intervals from one-dimensional profile likelihoods and study the coverage by generating several pseudo-experiments for a few benchmark sets of pseudo-true parameters. We use nested sampling to scan the parameter space and evaluate the coverage for the benchmarks when either flat or logarithmic priors are imposed on gaugino and scalar mass parameters. The sampling algorithm has been used in the configuration usually adopted for exploration of the Bayesian posterior. We observe both under- and over-coverage, which in some cases vary quite dramatically when benchmarks or priors are modified. We show how most of the variation can be explained as the impact of explicit priors as well as sampling effects, where the latter are indirectly imposed by physicality conditions. For comparison, we also evaluate the coverage for Bayesian credible intervals, and observe significant under-coverage in those cases.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Models of weak-scale supersymmetry offer viable dark matter (DM) candidates. | background 1.00 | 0.04 | 0.02 | asserted 0.81 |
| s1 | Their parameter spaces are however rather large and complex, such that pinning down the actual parameter values from experimental data can depend strongly on the employed statistical framework and scanning algorithm. | background 0.91 | 0.06 | 0.02 | asserted 0.93 |
| s2 | In frequentist parameter estimation, a central requirement for properly constructed confidence intervals is that they cover true parameter values, preferably at exactly the stated confidence level when experiments are repeated infinitely many times. | background 1.00 | 0.04 | 0.03 | asserted 0.97 |
| s3 | Since most widely-used scanning techniques are optimised for Bayesian statistics, one needs to assess their abilities in providing correct confidence intervals in terms of the statistical coverage. | gap 0.51 | 0.07 | 0.03 | asserted 0.91 |
| s4 | Here we investigate this for the Constrained Minimal Supersymmetric Standard Model (CMSSM) when only constrained by data from direct searches for dark matter. | contribution 0.87 | 0.80 | 0.03 | asserted 0.86 |
| s5 | We construct confidence intervals from one-dimensional profile likelihoods and study the coverage by generating several pseudo-experiments for a few benchmark sets of pseudo-true parameters. | method 0.96 | 0.96 | 0.04 | asserted 0.94 |
| s6 | We use nested sampling to scan the parameter space and evaluate the coverage for the benchmarks when either flat or logarithmic priors are imposed on gaugino and scalar mass parameters. | method 0.99 | 0.95 | 0.04 | asserted 0.96 |
| s7 | The sampling algorithm has been used in the configuration usually adopted for exploration of the Bayesian posterior. | method 0.92 | 0.53 | 0.03 | asserted 1.00 |
| s8 | We observe both under- and over-coverage, which in some cases vary quite dramatically when benchmarks or priors are modified. | result 1.00 | 0.96 | 0.41 | asserted 0.99 |
| s9 | We show how most of the variation can be explained as the impact of explicit priors as well as sampling effects, where the latter are indirectly imposed by physicality conditions. | interpretation 0.92 | 0.94 | 0.46 | asserted 1.00 |
| s10 | For comparison, we also evaluate the coverage for Bayesian credible intervals, and observe significant under-coverage in those cases. | result 1.00 | 0.97 | 0.12 | asserted 1.00 |
What was examined, and what was not
- arXiv listing abstract v2
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Models of weak-scale supersymmetry offer viable dark matter (DM) candidates. Their parameter spaces are however rather large and complex, such that pinning down the actual parameter values from experimental data can depend strongly on the employed statistical framework and scanning algorithm. In frequentist parameter estimation, a central requirement for properly constructed confidence intervals is that they cover true parameter values, preferably at exactly the stated confidence level when experiments are repeated infinitely many times. Since most widely-used scanning techniques are optimised for Bayesian statistics, one needs to assess their abilities in providing correct confidence intervals in terms of the statistical coverage. Here we investigate this for the Constrained Minimal Supersymmetric Standard Model (CMSSM) when only constrained by data from direct searches for dark matter. We construct confidence intervals from one-dimensional profile likelihoods and study the coverage by generating several pseudo-experiments for a few benchmark sets of pseudo-true parameters. We use nested sampling to scan the parameter space and evaluate the coverage for the benchmarks when either flat or logarithmic priors are imposed on gaugino and scalar mass parameters. The sampling algorithm has been used in the configuration usually adopted for exploration of the Bayesian posterior. We observe both under- and over-coverage, which in some cases vary quite dramatically when benchmarks or priors are modified. We show how most of the variation can be explained as the impact of explicit priors as well as sampling effects, where the latter are indirectly imposed by physicality conditions. For comparison, we also evaluate the coverage for Bayesian credible intervals, and observe significant under-coverage in those cases.
Human review
References
How the literature uses it
22 arXiv papers cite this in 31 sentences
Papers citing it · 18 sentences without a confident label
Uses its method or tool 2 papers
-
One can draw inference on a subset of $r≤n$ specific model parameters or observables, or a combination of both (collectively denoted by $ψ_i$), by “profiling” the likelihood along the other directions in the parameter space [ref] , [equation]
Dark matter and collider signatures of the MSSM · Statistical treatment of experimental constraints · 0.94
-
In spite of these small changes of the bounds depending on which exact priors and ranges one imposes on the parameters, our results show that one never obtains a $3σ$ bound on $λ$ larger than $∼1$.[In order to directly see that this $λ≲1$ is the least tight $3σ$ constraint on $λ$, we additionally performed a profile likelihood analysis of the parameter space; see, e.g., [ref] and references therein.
The landscape, the swampland and the era of precision cosmology · Dark energy and the cosmological data · 0.95
Compares with it 1 paper
-
In contrast, consistent (and sometimes drastic) under-coverage was observed [ref] for two different benchmark points reconstructed using mock ton-scale direct detection data.
Fundamental statistical limitations of future dark matter direct detection experiments · Comparison with other coverage studies · 0.97
Cites it as support 2 papers
-
[ref] , we believe that this under-coverage seen in the profile likelihood results, is due to sampling effects caused by strong prior effects that reduce the ability of our scanning algorithm (i.e. ) to explore regions at high masses and cross-sections. is a technique optimised for Bayesian statistics and, consequently, any frequentist inference it provides can be indirectly (and in some cases strongly) influenced by the prior effects.
How well will ton-scale dark matter direct detection experiments constrain minimal supersymmetry? · Bayesian credible and frequentist confidence regions · 0.97
-
Their stated coverage is also often somewhat incorrect, as they are computed based on asymptotic theory relying on Wilks' theorem, whose regularity assumptions are often violated in complicated parameter spaces such as the EWMSSM [ref] .
Combined collider constraints on neutralinos and charginos · Goodness-of-fit · 0.96
Cites it as background 6 papers
-
In this subsection, we review the basic concept of the Bayesian analysis [ref] (for its applications in physics, see e.g. Ref. [ref] , and for its recent applications in direct DM detection phenomenology, see e.g. Refs. [ref] )
Bayesian Reconstruction of the Velocity Distribution of Weakly Interacting Massive Particles from Direct Dark Matter Detection Data · Bayesian analysis · 0.99
-
[ref] and references therein.
How well will ton-scale dark matter direct detection experiments constrain minimal supersymmetry? · Statistical frameworks and scanning technique · 0.91
-
The coverage of parameter reconstructions has been previously discussed in the context of direct detection [ref] and collider identification [ref] of supersymmetric models.
Fundamental statistical limitations of future dark matter direct detection experiments · Introduction · 0.93
What it cites · 35 sentences without a confident label
Uses the method or tool of 9 papers
-
The particular algorithm is called [ref] and is implemented in [ref] (available from Ref. [ref] ), a publicly available package for exploring the CMSSM parameter space and comparing its predictions with different experimental data. includes many standalone packages as submodules which calculate different quantities for a given point in the CMSSM parameter space.
Implications for the Constrained MSSM from a new prediction for b to s gamma · Theoretical setup and scanning strategy · 1.00
-
The particular algorithm is called [ref] and is implemented in [ref] (available from Ref. [ref] ), a publicly available package for exploring the CMSSM parameter space and comparing its predictions with different experimental data. includes many standalone packages as submodules which calculate different quantities for a given point in the CMSSM parameter space.
MultiNest: an efficient and robust Bayesian inference tool for cosmology and particle physics · Theoretical setup and scanning strategy · 1.00
-
The particular algorithm is called [ref] and is implemented in [ref] (available from Ref. [ref] ), a publicly available package for exploring the CMSSM parameter space and comparing its predictions with different experimental data. includes many standalone packages as submodules which calculate different quantities for a given point in the CMSSM parameter space.
A Markov Chain Monte Carlo Analysis of the CMSSM · Theoretical setup and scanning strategy · 1.00
Uses data or values from 1 paper
-
We assume an allowed event energy range of 2-75 keV, take efficiencies from Ref. [ref] , and use an energy resolution of [ref] [equation] The 2 keV threshold of the XENON10 analysis presented in Ref. [ref] was based upon the understanding of the XENON10 detector behaviour at the time.
Constraints on inelastic dark matter from XENON10 · Direct detection likelihood, benchmarks and pseudo-experiments · 0.86
Cites as background 11 papers
-
This requirement for correct `coverage' serves as a powerful measure of success for any statistical method in extracting frequentist confidence intervals (for a discussion on statistical coverage, see e.g. Ref. [ref] ).
A Unified Approach to the Classical Statistical Analysis of Small Signals · Credible intervals, confidence intervals and statistical coverage · 0.88
-
The SUSY model we analyse here is the CMSSM [ref] .
Study of Constrained Minimal Supersymmetry · Theoretical setup and scanning strategy · 0.83
-
DD experiments aim to observe low energy nuclear recoils induced by the scattering of relic neutralinos off nuclei in the detector [ref] .
Supersymmetric Dark Matter · Direct detection likelihood, benchmarks and pseudo-experiments · 1.00
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.