A generalized Fourier approach to estimating the null parameters and proportion of non-null effects in large-scale multiple testing

In a recent paper (Efron (2004)), Efron pointed out that an important issue in large-scale multiple hypothesis testing is that the null distribution may be unknown and need to be estimated. Consider a Gaussian mixture model, where the null distribution is known to be normal but both null parameters--the mean and the variance--are unknown. We address the problem with a method based on Fourier transformation. The Fourier approach was first studied by Jin and Cai (2007), which focuses on the scenario where any non-null effect has either the same or a larger variance than that of the null effects. In this paper, we review the main ideas in Jin and Cai (2007), and propose a generalized Fourier approach to tackle the problem under another scenario: any non-null effect has a larger mean than that of the null effects, but no constraint is imposed on the variance. This approach and that in \cite{JC} complement with each other: each approach is successful in a wide class of situations where the other fails. Also, we extend the Fourier approach to estimate the proportion of non-null effects. The proposed procedures perform well both in theory and in simulated data.