Nonparametric f-Modeling for Empirical Bayes Inference with Unequal and Unknown Variances
Pith reviewed 2026-05-09 20:50 UTC · model grok-4.3
The pith
A generalized Tweedie identity recovers Bayes estimators and full posteriors for normal means with unknown unequal variances using only the observed marginal density.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that a generalized Tweedie-type identity expresses the Bayes estimator entirely in terms of the joint marginal density of the observed statistics and its partial derivatives, extending the classical formula to heterogeneous and unknown variances. Combined with a moment-generating-function representation inside the f-modeling paradigm, the identity recovers the full posterior distribution without specifying or estimating the prior. The resulting procedure supplies a unified nonparametric method for shrinkage estimation, uncertainty quantification, and hypothesis testing that accommodates arbitrary dependence between means and variances.
What carries the argument
The generalized Tweedie-type identity, which writes the Bayes estimator as a function of the joint marginal density and its partial derivatives.
If this is right
- Shrinkage estimates remain accurate when variances differ across observations.
- Full posterior distributions become available for uncertainty quantification without a parametric prior.
- Hypothesis testing and interval estimation can be performed inside the same nonparametric framework.
- Dependence between means and variances is handled without additional modeling assumptions.
- The approach extends classical empirical Bayes methods to the heteroscedastic setting common in real data.
Where Pith is reading between the lines
- The method could improve inference in fields such as genomics or imaging where observation noise levels vary systematically.
- Density estimation error in moderate sample sizes might require careful bandwidth or smoothing choices to maintain reliability.
- The framework suggests a route to adaptive multiple-testing procedures that automatically account for variance heterogeneity.
- Extensions to non-normal sampling distributions would require analogous marginal-density identities.
Load-bearing premise
The joint marginal density of the observed statistics and its partial derivatives can be estimated nonparametrically from finite samples with enough accuracy to recover the posterior quantities reliably.
What would settle it
In a simulation study where the true prior and heterogeneous variances are known, the nonparametric estimator of the posterior mean deviates substantially from the oracle Bayes estimator computed directly from the known prior.
Figures
read the original abstract
Empirical Bayes methods are widely used for large-scale inference, yet most classical approaches assume homoscedastic observations and focus primarily on posterior mean estimation. We develop a nonparametric empirical Bayes framework for the heteroscedastic normal means problem with unequal and unknown variances. Our first contribution is a generalized Tweedie-type identity that expresses the Bayes estimator entirely in terms of the joint marginal density of the observed statistics and its partial derivatives, extending the classical Tweedie's formula to settings with heterogeneous and unknown variances. Our second contribution is to introduce a moment-generating-function representation that enables recovery of the full posterior distribution within the f-modeling paradigm without specifying or estimating the prior distribution. The resulting method provides a unified framework for point estimation, uncertainty quantification, and hypothesis testing while accommodating arbitrary dependence between means and variances. Simulation studies and real-data analysis demonstrate that the proposed approach achieves accurate shrinkage estimation and reliable posterior inference in heterogeneous data environments.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript develops a nonparametric f-modeling framework for empirical Bayes inference in the heteroscedastic normal means problem with unequal and unknown variances. The central contributions are a generalized Tweedie-type identity that writes the Bayes estimator (and, via MGF, the full posterior) solely in terms of the joint marginal density of the observed statistics and variance estimates together with its partial derivatives, and a practical estimation procedure that avoids specifying or estimating the prior. Simulation studies and a real-data example are presented to illustrate performance for shrinkage estimation, uncertainty quantification, and testing.
Significance. If the generalized identity is correctly derived and the nonparametric plug-in estimators prove stable, the work supplies a unified, prior-free approach to full posterior inference under arbitrary mean-variance dependence. This directly addresses a common practical limitation of classical EB methods and could be useful in large-scale applications with heterogeneous precision. The f-modeling route and MGF representation are genuine strengths when the density estimation step succeeds.
major comments (2)
- [Methodology / estimation procedure] The central claim rests on accurate nonparametric recovery of the joint marginal density of (X, variance estimate) and its first- and higher-order partial derivatives. No convergence rates, bias bounds, or robustness analysis are supplied for these estimators under the heteroscedastic model with unknown variances and possible mean-variance dependence; this step is load-bearing for consistency of the resulting posterior quantities.
- [Simulation studies and real-data analysis] The simulation design and real-data analysis are described only at a high level. Specific regimes with strong mean-variance dependence, small n, or high-dimensional covariate effects on variance should be added to test whether the plug-in estimators remain stable outside the reported settings.
minor comments (2)
- Notation for the observed statistics and the variance estimator should be introduced once and used consistently; the distinction between the true variance and its estimate is occasionally blurred in the abstract and early sections.
- [Abstract] The abstract states that the method 'accommodates arbitrary dependence between means and variances,' but the precise smoothness or support conditions required for the partial derivatives to exist are not stated explicitly.
Simulated Author's Rebuttal
We thank the referee for the detailed and constructive comments on our manuscript. We address each of the major comments below and describe the changes we plan to make in the revised version.
read point-by-point responses
-
Referee: [Methodology / estimation procedure] The central claim rests on accurate nonparametric recovery of the joint marginal density of (X, variance estimate) and its first- and higher-order partial derivatives. No convergence rates, bias bounds, or robustness analysis are supplied for these estimators under the heteroscedastic model with unknown variances and possible mean-variance dependence; this step is load-bearing for consistency of the resulting posterior quantities.
Authors: We agree that providing some analysis of the estimation procedure's properties would strengthen the paper. While the manuscript prioritizes the derivation of the generalized Tweedie identity and the MGF-based posterior recovery, we will revise the methodology section to include a discussion of the consistency of the nonparametric estimators. Specifically, we will reference standard results on the convergence of kernel density estimators in multiple dimensions and discuss how the rates depend on the bandwidth choice and the smoothness of the joint density. Additionally, we will add a robustness study in the simulations to examine performance under mean-variance dependence. revision: yes
-
Referee: [Simulation studies and real-data analysis] The simulation design and real-data analysis are described only at a high level. Specific regimes with strong mean-variance dependence, small n, or high-dimensional covariate effects on variance should be added to test whether the plug-in estimators remain stable outside the reported settings.
Authors: We appreciate this recommendation. The original simulations were designed to cover a range of heteroscedastic settings, but we acknowledge that more extreme cases could better validate the method. In the revised manuscript, we will expand the simulation section with additional experiments including strong mean-variance dependence, smaller sample sizes (e.g., n=100), and cases where variances depend on high-dimensional covariates. We will also provide more detailed descriptions of the data-generating processes and include further analysis of the real-data example to highlight performance in practical scenarios. revision: yes
Circularity Check
No significant circularity in derivation chain
full rationale
The paper's core contribution is a mathematically derived generalized Tweedie identity expressing the Bayes estimator via the joint marginal density and its partial derivatives; this is an extension of a standard identity rather than a self-definition or fitted-input renaming. The MGF representation for full posterior recovery is likewise obtained directly from the estimated marginal within the f-modeling framework, without reducing to prior specification or self-citation chains. No load-bearing steps equate outputs to inputs by construction, and the approach remains self-contained against external mathematical and nonparametric estimation benchmarks.
Axiom & Free-Parameter Ledger
axioms (2)
- domain assumption Observations are independent normal random variables with unknown means and variances.
- domain assumption The joint marginal density of the observed statistics is sufficiently smooth to admit partial derivatives.
Reference graph
Works this paper leans on
-
[1]
Journal of Statistical Planning and Inference , volume=
Adapting to one- and two-way classified structures of hypotheses while controlling the false discovery rate , author=. Journal of Statistical Planning and Inference , volume=. 2021 , publisher=
work page 2021
-
[2]
Asymptotic properties of sufficient dimension reduction with a diverging number of predictors , author=. Statistica Sinica , volume=. 2011 , publisher=
work page 2011
-
[3]
Handbook of Bayesian, Fiducial, and Frequentist Inference , pages=
Empirical bayes: Concepts and methods , author=. Handbook of Bayesian, Fiducial, and Frequentist Inference , pages=. 2024 , publisher=
work page 2024
-
[4]
Journal of the American Statistical Association , volume=
Nonparametric maximum likelihood estimation of a mixing distribution , author=. Journal of the American Statistical Association , volume=. 1978 , publisher=
work page 1978
-
[5]
Journal of Applied Econometrics , volume=
Empirical Bayesball remixed: Empirical Bayes methods for longitudinal data , author=. Journal of Applied Econometrics , volume=. 2017 , publisher=
work page 2017
-
[6]
Robust empirical Bayes confidence intervals , author=. Econometrica , volume=. 2022 , publisher=
work page 2022
-
[7]
Where to find needles in a haystack? , author=. TEST , volume=. 2022 , publisher=
work page 2022
-
[8]
Computational statistics , volume=
Type S error rates for classical and Bayesian single and multiple comparison procedures , author=. Computational statistics , volume=. 2000 , publisher=
work page 2000
-
[9]
False discovery rates: a new deal , author=. Biostatistics , volume=. 2017 , publisher=
work page 2017
-
[10]
arXiv preprint arXiv:2402.08792 , year=
Interpretation of local false discovery rates under the zero assumption , author=. arXiv preprint arXiv:2402.08792 , year=
-
[11]
Estimating the false discovery rate using nonparametric deconvolution , author=. Biometrics , volume=. 2007 , publisher=
work page 2007
-
[12]
Journal of the American Statistical Association , volume=
Convex optimization, shape constraints, compound decisions, and empirical Bayes rules , author=. Journal of the American Statistical Association , volume=. 2014 , publisher=
work page 2014
-
[13]
Saremi, Saeed and Hyv. Neural empirical. Journal of Machine Learning Research , volume=
-
[14]
Science of the Total Environment , volume=
Empirical Bayesian kriging implementation and usage , author=. Science of the Total Environment , volume=. 2020 , publisher=
work page 2020
-
[15]
Harmonization of cortical thickness measurements across scanners and sites , author=. Neuroimage , volume=. 2018 , publisher=
work page 2018
-
[16]
An empirical Bayes approach to inferring large-scale gene association networks , author=. Bioinformatics , volume=. 2005 , publisher=
work page 2005
-
[17]
Exploring the information in p-values for the analysis and planning of multiple-test experiments , author=. Biometrics , volume=. 2007 , publisher=
work page 2007
-
[18]
Comment: Microarrays, Empirical Bayes and the Two-Groups Model , author=. Statistical Science , volume=
-
[19]
The origins of sex differences in science , author=. Social forces , volume=. 1990 , publisher=
work page 1990
-
[20]
Journal of the Royal Statistical Society: Series A (General) , volume=
Generalized linear models , author=. Journal of the Royal Statistical Society: Series A (General) , volume=. 1972 , publisher=
work page 1972
-
[21]
Martin, R. and Tokdar, S. T. , journal=. A nonparametric empirical. 2012 , publisher=
work page 2012
-
[22]
The optimal power puzzle: scrutiny of the monotone likelihood ratio assumption in multiple testing , author=. Biometrika , volume=. 2013 , publisher=
work page 2013
- [23]
-
[24]
Bias, Regret and Statistical Inference in Adaptive Data Collection , author=. 2022 , note=
work page 2022
-
[25]
Resampling-based Bias Adjustment for Adaptively Collected Data , author=. 2022 , note=
work page 2022
-
[26]
G. E. P. Box , title =. Biometrika , volume =. 1953 , pages =. doi:10.1093/biomet/40.3-4.318 , masid =
-
[27]
Kwon, Yeil and Zhao, Zhigen , year=. On
-
[28]
R. Nuzzo. Scientific method: Statistical errors. Nature. 2014
work page 2014
-
[29]
J. P. A. Ioannidis. Why most discovered true associations are inflated. Epidemiology. 2008
work page 2008
-
[30]
Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume=
Covariate-assisted ranking and screening for large-scale two-sample inference , author=. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume=. 2019 , publisher=
work page 2019
-
[31]
Journal of the American Statistical Association , note=
Admissibility in partial conjunction testing , author=. Journal of the American Statistical Association , note=. 2017 , publisher=
work page 2017
-
[32]
Kwon, Y., and Zhao, Z. , year=. Nonparametric Empirical
-
[33]
1,500 scientists lift the lid on reproducibility , author=. Nature News , volume=
- [34]
-
[35]
Proceedings of the Royal Society of London , volume=
Note on regression and inheritance in the case of two parents , author=. Proceedings of the Royal Society of London , volume=. 1895 , publisher=
-
[36]
The annals of mathematical statistics , pages=
A non-parametric test of independence , author=. The annals of mathematical statistics , pages=. 1948 , publisher=
work page 1948
-
[37]
Measuring statistical dependence with Hilbert-Schmidt norms , author=. ALT , volume=. 2005 , organization=
work page 2005
-
[38]
The annals of statistics , pages=
On nonparametric measures of dependence for random variables , author=. The annals of statistics , pages=. 1981 , publisher=
work page 1981
-
[39]
The annals of statistics , volume=
Measuring and testing dependence by correlation of distances , author=. The annals of statistics , volume=. 2007 , publisher=
work page 2007
-
[40]
Journal of Machine Learning Research , volume=
Consistent nonparametric tests of independence , author=. Journal of Machine Learning Research , volume=
-
[41]
Empirical Bayes and compound estimation of normal means , author=. Statistica Sinica , volume=. 1997 , publisher=
work page 1997
-
[42]
arXiv preprint arXiv:1610.05246 , year=
BET on Independence , author=. arXiv preprint arXiv:1610.05246 , year=
-
[43]
arXiv preprint arXiv:1608.07885 , year=
Fisher exact scanning for dependency , author=. arXiv preprint arXiv:1608.07885 , year=
-
[44]
Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume=
Sure independence screening for ultrahigh dimensional feature space , author=. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume=. 2008 , publisher=
work page 2008
-
[45]
McAuliffe, J. D. and Blei, D. M. and Jordan, M. I. , journal=. Nonparametric empirical. 2006 , publisher=
work page 2006
-
[46]
Liu, J. S. , journal=. Nonparametric hierarchical. 1996 , publisher=
work page 1996
-
[47]
Winter, B. and Grawunder, S. , journal=. The phonetic profile of. 2012 , publisher=
work page 2012
-
[48]
The review of economic studies , volume=
Some tests of specification for panel data: Monte Carlo evidence and an application to employment equations , author=. The review of economic studies , volume=. 1991 , publisher=
work page 1991
-
[49]
Journal of the American Statistical Association , volume=
A factor model approach to multiple testing under dependence , author=. Journal of the American Statistical Association , volume=. 2009 , publisher=
work page 2009
-
[50]
Liu, J. and Pearlson, G. and Windemuth, A. and Ruano, G. and Perrone-Bizzozero, N. I. and Calhoun, V. , journal=. Combining f. 2009 , publisher=
work page 2009
-
[51]
Lin, D. and Calhoun, V. D. and Wang, Y. , journal=. Correspondence between f. 2014 , publisher=
work page 2014
-
[52]
An efficient multi-locus mixed-model approach for genome-wide association studies in structured populations , author=. Nature Genetics , volume=. 2012 , publisher=
work page 2012
-
[53]
Proceedings of the National Academy of Sciences of the United States of America , volume=
Significance analysis of time course microarray experiments , author=. Proceedings of the National Academy of Sciences of the United States of America , volume=. 2005 , publisher=
work page 2005
-
[54]
Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume=
Strong control, conservative point estimation and simultaneous conservative consistency of false discovery rates: a unified approach , author=. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , volume=. 2004 , publisher=
work page 2004
-
[55]
Covariate-assisted ranking and screening for large-scale two-sample inference , author=
-
[56]
Journal of Machine Learning Research , volume=
Adaptive false discovery rate control under independence and dependence , author=. Journal of Machine Learning Research , volume=
-
[57]
Singh, A. K. and Phillips, S. , journal=. Hierarchical control of false discovery rate for phase locking measures of. 2010 , publisher=
work page 2010
-
[58]
Detecting differential gene expression with a semiparametric hierarchical mixture method , author=. Biostatistics , volume=. 2004 , publisher=
work page 2004
-
[59]
More specific signal detection in functional magnetic resonance imaging by false discovery rate control for hierarchically structured systems of hypotheses , author=. PloS one , volume=. 2016 , publisher=
work page 2016
-
[60]
A two-step hierarchical hypothesis set testing framework, with applications to gene expression data on ordered categories , author=. BMC Bioinformatics , volume=. 2014 , publisher=
work page 2014
-
[61]
Journal of the American Statistical Association , volume=
Hierarchical false discovery rate--controlling methodology , author=. Journal of the American Statistical Association , volume=. 2008 , publisher=
work page 2008
- [62]
-
[63]
He, L. and Sarkar, S. K. , journal=. On improving some adaptive. 2013 , publisher=
work page 2013
-
[64]
Sarkar, S. K. and Heller, R. , journal=. Comments on:. 2008 , publisher=
work page 2008
-
[65]
Sarkar, S. K. , booktitle=. On the. 2008 , publisher=
work page 2008
-
[66]
Sarkar, S. K. , journal=. Two-stage stepup procedures controlling. 2008 , publisher=
work page 2008
-
[67]
Gavrilov, Y. and Benjamini, Y. and Sarkar, S. K. , journal=. An adaptive step-down procedure with proven. 2009 , publisher=
work page 2009
-
[68]
Sarkar, S. K. , journal=. Some probability inequalities for ordered. 1998 , publisher=
work page 1998
-
[69]
Sarkar, S. K. , journal=. The. 1997 , publisher=
work page 1997
-
[70]
Neal, R. M. and Hinton, G. E. , booktitle=. A view of the. 1998 , publisher=
work page 1998
-
[71]
Dempster, A. P. and Laird, N. M. and Rubin, D. B. , journal=. Maximum likelihood from incomplete data via the. 1977 , publisher=
work page 1977
-
[72]
Journal of Statistical Software , volume=
structSSI: Simultaneous and Selective Inference for Grouped or Hierarchically Structured Data , author=. Journal of Statistical Software , volume=. 2014 , publisher=
work page 2014
-
[73]
Data-driven hypothesis weighting increases detection power in genome-scale multiple testing , author=. Nature Methods , volume=. 2016 , publisher=
work page 2016
-
[74]
Genetic Epidemiology , volume=
Stratified false discovery control for large-scale hypothesis testing with application to genome-wide association studies , author=. Genetic Epidemiology , volume=. 2006 , publisher=
work page 2006
-
[75]
Ferkingstad, E. and Frigessi, A. and Rue, H. and Thorleifsson, G. and Kong, A. , journal=. Unsupervised empirical. 2008 , publisher=
work page 2008
-
[76]
Journal of the Royal Statistical Society: Series B , year=
The p-filter: multilayer false discovery rate control for grouped hypotheses , author=. Journal of the Royal Statistical Society: Series B , year=
-
[77]
Genome-wide meta-analysis increases to 71 the number of confirmed Crohn's disease susceptibility loci , author=. Nature Genetics , volume=. 2010 , publisher=
work page 2010
-
[78]
Journal of the American Statistical Association , volume=
False discovery rate regression: an application to neural synchrony detection in primary visual cortex , author=. Journal of the American Statistical Association , volume=. 2015 , publisher=
work page 2015
-
[79]
Covariate-modulated local false discovery rate for genome-wide association studies , author=. Bioinformatics , pages=. 2014 , publisher=
work page 2014
-
[80]
Genetic epidemiology , volume=
Many Phenotypes Without Many False Discoveries: Error Controlling Strategies for Multitrait Association Studies , author=. Genetic epidemiology , volume=. 2016 , publisher=
work page 2016
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.