Evidence for Conservatism in LHC SUSY Searches

The standard in the high energy physics community for claiming discovery of new physics is a $5\sigma$ excess in the observed signal over the estimated background. While a $3\sigma$ excess is not enough to claim discovery, it is certainly enough to pique the interest of both experimentalists and theorists. However, with a large number of searches performed by both the ATLAS and CMS collaborations at the LHC, one expects a nonzero number of multi-$\sigma$ results simply due to statistical fluctuations in the no-signal scenario. Our analysis examines the distribution of p-values for CMS and ATLAS supersymmetry (SUSY) searches using the full 2011 data set to determine if the collaborations are being overly conservative in their analyses. We find that there is a statistically significant excess of `medium' $\sigma$ values at the level of $p=0.005$, indicating over-conservativism in the estimation of uncertainties.