Blind search for the real sample: Application to the origin of ultra-high energy cosmic rays

We suggest a method for statistical tests which does not suffer from a posteriori manipulations with tested samples (e.g. cuts optimization) and does not require a somewhat obscure procedure of the penalty estimate. The idea of the method is to hide the real sample (before it has been studied) among a large number of artificial samples, drawn from a random distribution expressing the null hypothesis, and then to search for it as the one demonstrating the strongest hypothesized effect. The statistical significance of the effect in this approach is the inverse of the maximal number of random samples at which the search was successful. We have applied the method to revisit the problem of correlation between the arrival directions of ultra-high energy cosmic rays and BL Lac objects. No significant correlation was found.