pith. sign in

arxiv: 2411.03304 · v3 · submitted 2024-11-05 · 📊 stat.ME · math.ST· stat.TH

Bayesian Controlled FDR Variable Selection via Parameter-Expanded Latent Knockoffs

Lorenzo Focardi-Olmi , Anna Gottard , Michele Guindani , Marina Vannucci This is my paper

Pith reviewed 2026-05-23 17:44 UTC · model grok-4.3

classification 📊 stat.ME math.STstat.TH
keywords Bayesian variable selectionknockoff filterfalse discovery rateGaussian graphical modelspike-and-slab priormodel-X knockoffslatent variables
0
0 comments X

The pith

A parameter-expanded latent knockoff layer in a Bayesian joint model controls the FDR at any chosen level for known covariate distributions.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper develops a Bayesian generalization of the model-X knockoff filter for normally distributed covariates by embedding a latent knockoff layer in a joint model for covariates and response. A Gaussian graphical model captures the conditional independence structure among covariates and informs the construction of the latent layer through parameter expansion of the response model. Selection proceeds via an upper bound on the posterior probability of non-inclusion under a modified spike-and-slab prior that keeps model dimension fixed. When the covariate distribution is fully known the procedure guarantees finite-sample control of the Bayesian FDR at any target level; an estimated graph yields an asymptotic guarantee. Simulations indicate higher selection stability than classical knockoff methods while matching or exceeding the performance of other Bayesian variable selection approaches.

Core claim

The induced latent knockoff layer defines valid Gaussian model-X knockoffs under the proposed construction and that the resulting procedure controls the Bayesian FDR at an arbitrary level, in finite samples, if the distribution of the covariates is fully known; under an estimated graphical structure, it satisfies an asymptotic FDR guarantee. The model performs variable selection using an upper bound on the posterior probability of non-inclusion and addresses extensions to non-Gaussian responses.

What carries the argument

The parameter-expanded latent knockoff layer, built from the Gaussian graphical model of the covariates, that supplies valid auxiliary variables inside the response model without increasing its dimension.

If this is right

  • The procedure controls Bayesian FDR at any arbitrary level in finite samples when the covariate distribution is known.
  • An estimated graphical structure yields asymptotic FDR control.
  • Variable selection stability increases relative to classical knockoff methods in simulations.
  • Performance is comparable or better than standard Bayesian variable selection while maintaining FDR control.
  • The construction extends to non-Gaussian responses.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The latent-layer construction could be adapted to other families of graphical models for the covariates.
  • The framework may allow direct incorporation of prior information on covariate dependence structures in applied settings.
  • Hybrid procedures that blend the Bayesian FDR control with frequentist knockoff thresholds could be examined.
  • The method's performance under moderate graph misspecification remains a natural next test case.

Load-bearing premise

The covariates follow a multivariate normal distribution that lets the parameter-expanded representation generate valid model-X knockoffs.

What would settle it

A simulation with fully known but non-Gaussian covariate distributions in which the realized Bayesian FDR exceeds the nominal target level.

Figures

Figures reproduced from arXiv: 2411.03304 by Anna Gottard, Lorenzo Focardi-Olmi, Marina Vannucci, Michele Guindani.

Figure 1
Figure 1. Figure 1: Simulation study: Scale-free graph structure under Gaussian graphical model in Scenario 2. White nodes correspond to the noise variables, and blue nodes to the active ones. A stronger blue color corresponds to a larger effect. and Peterson et al. (2016). Specifically, we generated the covariates from a graph with 40 hubs, each with 5 connected nodes. This resulted in 240 covariates with a sparse graphical … view at source ↗
Figure 2
Figure 2. Figure 2: Simulation study: Variable selection performed by our Bayesian knockoff filter method (BayeKnock) and by spike-and-regression (Spike-Slab) on a simulated dataset under Scenario 3. Top plot (BayesKnock): Bars represent estimates of the upper bound for P[rj = 0 | D] as in Equation (17) while points represent the estimated BFDR. Red points correspond to true active variables. Bottom plot (Spike-Slab): Estimat… view at source ↗
Figure 3
Figure 3. Figure 3: Prostate cancer data: Posterior distribution of Wj , j = 1, . . . , 6. Those high￾lighted in blue are selected according to [PITH_FULL_IMAGE:figures/full_fig_p030_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Prostate cancer data: Variable selection. Bars represent estimates 2 Pb[Wj ≤ 0 | D], j = 1, . . . 6, in increasing order and points represent the estimated BFDR(S) with S the set of all previous indexes. The dashed red line is the chosen threshold q = 0.1. We also applied the state-of-art procedures used for comparison in the simulation study [PITH_FULL_IMAGE:figures/full_fig_p031_4.png] view at source ↗
read the original abstract

In many research fields, researchers aim to identify significant associations between a set of explanatory variables and a response while controlling the FDR. The Knockoff filter has been recently proposed in the frequentist paradigm to introduce controlled noise in a model by cleverly constructing copies of the predictors as auxiliary variables. We develop a fully Bayesian generalization of the classical model-X knockoff filter for normally distributed covariates. In our approach, we consider a joint model for the covariates and the response, where the conditional independence structure of the covariates is captured through a Gaussian graphical model and used to define a latent knockoff layer through a parameter-expanded representation of the response model. Estimating the covariate graph informs the knockoff construction and improves inference on the covariate effects. We use a modified spike-and-slab prior on the regression coefficients, avoiding the increase of the model dimension typical of the classical knockoff filter. We also address extensions to non-Gaussian responses. Our model performs variable selection using an upper bound on the posterior probability of non-inclusion. We show that the induced latent knockoff layer defines valid Gaussian model-X knockoffs under the proposed construction and that the resulting procedure controls the Bayesian FDR at an arbitrary level, in finite samples, if the distribution of the covariates is fully known; under an estimated graphical structure, it satisfies an asymptotic FDR guarantee. We use simulated data to demonstrate that our proposal increases the stability of the selection with respect to classical knockoff methods. With respect to Bayesian variable selection methods, our selection procedure achieves comparable or better performances, while maintaining control over the FDR. We conclude with an application to real data.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper proposes a Bayesian generalization of the model-X knockoff filter for variable selection under FDR control. It models jointly Gaussian covariates via a Gaussian graphical model and the response, introducing a parameter-expanded latent knockoff layer that embeds knockoff generation inside the response model. A modified spike-and-slab prior is used on regression coefficients to avoid dimension inflation. The central claim is that the induced latent layer produces valid model-X knockoffs, yielding exact finite-sample Bayesian FDR control when the covariate distribution is known exactly, and an asymptotic guarantee when the graph is estimated from data. Simulations demonstrate improved selection stability relative to classical knockoffs while maintaining FDR control; an application to real data is included. Extensions to non-Gaussian responses are sketched.

Significance. If the finite-sample FDR guarantee holds via the latent construction, the work would provide a novel Bayesian route to exact FDR control that integrates graphical structure estimation directly into selection without the usual auxiliary-variable dimension increase. The parameter-expansion device and the use of an upper bound on posterior non-inclusion probability are technically interesting and could influence subsequent Bayesian knockoff-style methods.

major comments (2)
  1. [Abstract, §3] Abstract and §3 (theoretical results): the finite-sample Bayesian FDR control is asserted to follow once the latent knockoff layer defines valid Gaussian model-X knockoffs. However, the derivation does not explicitly verify that the parameter-expanded joint distribution satisfies both the exchangeability (swap) property and the conditional independence property required by the model-X framework when the covariates are multivariate normal; this step is load-bearing for the exact finite-sample claim.
  2. [§4] §4 (asymptotic guarantee under estimated graph): the asymptotic FDR control is stated to hold when the graphical structure is estimated, but no rate or consistency condition on the graph estimator is supplied that would guarantee the knockoff properties are preserved asymptotically; without this, the transition from the known-distribution case to the estimated-graph case is not fully justified.
minor comments (2)
  1. [§2] Notation for the latent knockoff variables and the parameter-expansion matrix should be introduced with a single consistent symbol set in §2 to avoid later confusion.
  2. [Simulations] Simulation section: the data-generating process for the non-null coefficients and the exact FDR target levels used in the tables should be stated explicitly rather than referenced only in the caption.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and constructive comments. We address the two major comments point by point below.

read point-by-point responses

  1. Referee: [Abstract, §3] Abstract and §3 (theoretical results): the finite-sample Bayesian FDR control is asserted to follow once the latent knockoff layer defines valid Gaussian model-X knockoffs. However, the derivation does not explicitly verify that the parameter-expanded joint distribution satisfies both the exchangeability (swap) property and the conditional independence property required by the model-X framework when the covariates are multivariate normal; this step is load-bearing for the exact finite-sample claim.

    Authors: We appreciate the referee's observation. Section 3 derives that the parameter-expanded latent knockoff construction yields valid model-X knockoffs for multivariate normal covariates by leveraging the symmetry of the Gaussian graphical model and the latent layer's separation of knockoff generation. The exchangeability (swap) property holds because the joint precision matrix remains invariant under the relevant variable permutations induced by the graph, and the conditional independence property follows from the latent variables being independent of the response given the original covariates. To make this verification fully explicit and address the concern, we will add a dedicated lemma in the revised §3 that directly verifies both properties under the parameter-expanded representation. revision: yes

  2. Referee: [§4] §4 (asymptotic guarantee under estimated graph): the asymptotic FDR control is stated to hold when the graphical structure is estimated, but no rate or consistency condition on the graph estimator is supplied that would guarantee the knockoff properties are preserved asymptotically; without this, the transition from the known-distribution case to the estimated-graph case is not fully justified.

    Authors: The referee correctly identifies that the asymptotic claim in §4 would be strengthened by explicit conditions. In the revision we will state that the graph estimator must be consistent for the true precision matrix (e.g., satisfying the high-dimensional rates of Meinshausen-Bühlmann or Ravikumar et al. under suitable sparsity and sample-size regimes) so that the induced knockoff distribution converges to the oracle distribution, thereby preserving the asymptotic FDR control. This clarification will be incorporated into the statement and proof sketch of the asymptotic result. revision: yes

Circularity Check

0 steps flagged

No circularity: FDR control follows from explicit knockoff validity under known Gaussian covariates

full rationale

The paper constructs a parameter-expanded latent knockoff layer inside a joint Gaussian graphical model for covariates and response, then asserts that this layer satisfies model-X exchangeability and conditional independence. The finite-sample Bayesian FDR control is stated to follow directly from that construction when the covariate distribution is known exactly (with an asymptotic guarantee when the graph is estimated). No equation in the abstract or described derivation reduces the FDR bound to a fitted parameter by construction, renames a known result, or relies on a load-bearing self-citation whose content is itself unverified. The guarantee is conditional on an external modeling assumption rather than tautological with the fitted quantities. This is the normal case of a self-contained derivation.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 1 invented entities

The central claim rests on the domain assumption of multivariate normal covariates and introduces the latent knockoff layer as a new modeling device without external falsifiable evidence beyond the construction itself.

axioms (1)
  • domain assumption Covariates follow a multivariate normal distribution whose conditional independence structure is captured by a Gaussian graphical model.
    Invoked to define valid Gaussian model-X knockoffs and the latent layer.
invented entities (1)
  • latent knockoff layer no independent evidence
    purpose: To enable Bayesian FDR control without increasing model dimension
    Introduced via parameter expansion of the response model; no independent evidence supplied outside the paper.

pith-pipeline@v0.9.0 · 5830 in / 1331 out tokens · 34316 ms · 2026-05-23T17:44:14.694891+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

54 extracted references · 54 canonical work pages

  1. [1]

    and Chib, S

    Albert, J. and Chib, S. (1993). B ayesian analysis of binary and polychotomous response data. Journal of American Statistical Association , 88:669--679

    work page 1993

  2. [2]

    Barber, R. F. and Cand \`e s, E. (2015). Controlling the false discovery rate via knockoffs . The Annals of Statistics , 43(5):2055 -- 2085

    work page 2015

  3. [3]

    Barber, R. F. and Cand \`e s, E. (2019). A knockoff filter for high-dimensional selective inference. The Annals of Statistics , 47(5):2504--2537

    work page 2019

  4. [4]

    F., Candès, E., and Samworth, R

    Barber, R. F., Candès, E., and Samworth, R. J. (2020). Robust inference with knockoffs . The Annals of Statistics , 48(3):1409 -- 1431

    work page 2020

  5. [5]

    Barbieri, M. M. and Berger, J. O. (2004). Optimal predictive model selection. The Annals of Statistics , 32(3):870 -- 897

    work page 2004

  6. [6]

    Bates, S., Candès, E., Janson, L., and Wang, W. (2021). Metropolized knockoff sampling. Journal of the American Statistical Association , 116(535):1413--1427

    work page 2021

  7. [7]

    and Hochberg, Y

    Benjamini, Y. and Hochberg, Y. (1995). Controlling the false discovery rate: A practical and powerful approach to multiple testing. Journal of the Royal Statistical Society: Series B (Methodological) , 57(1):289--300

    work page 1995

  8. [8]

    and Yekutieli, D

    Benjamini, Y. and Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. The Annals of Statistics , 29(4):1165--1188

    work page 2001

  9. [9]

    and Yekutieli, D

    Benjamini, Y. and Yekutieli, D. (2005). False discovery rate–adjusted multiple confidence intervals for selected parameters. Journal of the American Statistical Association , 100(469):71--81

    work page 2005

  10. [10]

    Berti, P., Dreassi, E., Leisen, F., Pratelli, L., and Rigo, P. (2023). New perspectives on knockoffs construction. Journal of Statistical Planning and Inference , 223:1--14

    work page 2023

  11. [11]

    M., Kucukelbir, A., and McAuliffe, J

    Blei, D. M., Kucukelbir, A., and McAuliffe, J. D. (2017). Variational inference: A review for statisticians. Journal of the American statistical Association , 112(518):859--877

    work page 2017

  12. [12]

    Brooks, S., Gelman, A., Jones, G., and Meng, X.-L. (2011). Handbook of M arkov Chain Monte Carlo . CRC press

    work page 2011

  13. [13]

    J., Vannucci, M., and Fearn, T

    Brown, P. J., Vannucci, M., and Fearn, T. (1998). Multivariate bayesian variable selection and prediction. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 60(3):627--641

    work page 1998

  14. [14]

    Candès, E., Fan, Y., Janson, L., and Lv, J. (2018). Panning for gold: ‘model-x’ knockoffs for high dimensional controlled variable selection. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 80(3):551--577

    work page 2018

  15. [15]

    M., Polson, N

    Carvalho, C. M., Polson, N. G., and Scott, J. G. (2010). The horseshoe estimator for sparse signals . Biometrika , 97(2):465--480

    work page 2010

  16. [16]

    and Roquain, \'E

    Castillo, I. and Roquain, \'E . (2020). On spike and slab empirical B ayes multiple testing. The Annals of Statistics , 48(5):2544--2574

    work page 2020

  17. [17]

    Dai, C., Lin, B., Xing, X., and Liu, J. S. (2022). False discovery rate control via data splitting. Journal of the American Statistical Association , 118(544):2503--2520

    work page 2022

  18. [18]

    Dreassi, E., Leisen, F., Pratelli, L., and Rigo, P. (2024). Generating knockoffs via conditional independence. Electronic Journal of Statistics , 18(1):119--144

    work page 2024

  19. [19]

    and Li, R

    Fan, J. and Li, R. (2001). Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association , 96(456):1348--1360

    work page 2001

  20. [20]

    Friedman, J., Hastie, T., and Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical LASSO . Biostatistics , 9(3):432--441

    work page 2008

  21. [21]

    George, E. I. and McCulloch, R. E. (1997). Approaches for B ayesian variable selection. Statistica Sinica , 7(2):339--373

    work page 1997

  22. [22]

    Geweke, J. (1992). Evaluating the accuracy of sampling-based approaches to the calculations of posterior moments. Bayesian Statistics , 4:641--649

    work page 1992

  23. [23]

    and Yin, G

    Gu, J. and Yin, G. (2021). B ayesian knockoff filter using G ibbs sampler. arXiv preprint arXiv:2102.05223

    work page arXiv 2021

  24. [24]

    Guindani, M., M \"u ller, P., and Zhang, S. (2009). A B ayesian discovery procedure. Journal of the Royal Statistical Society Series B: Statistical Methodology , 71(5):905--925

    work page 2009

  25. [25]

    and Li, H

    Li, C. and Li, H. (2008). Network-constrained regularization and variable selection for analysis of genomic data. Bioinformatics , 24(9):1175--1182

    work page 2008

  26. [26]

    Liu, H., Lafferty, J., and Wasserman, L. (2009). The N onparanormal: Semiparametric estimation of high dimensional undirected graphs. Journal of Machine Learning Research , 10:2295--2328

    work page 2009

  27. [27]

    and B \"u hlmann, P

    Meinshausen, N. and B \"u hlmann, P. (2010). Stability selection. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 72(4):417--473

    work page 2010

  28. [28]

    and Bühlmann, P

    Meinshausen, N. and Bühlmann, P. (2006). High-dimensional graphs and variable selection with the lasso. The Annals of Statistics , 34(3):1436--1462

    work page 2006

  29. [29]

    Mitchell, T. J. and Beauchamp, J. J. (1988). B ayesian variable selection in linear regression. Journal of the American Statistical Association , 83(404):1023--1032

    work page 1988

  30. [30]

    Muller, P., Parmigiani, G., and Rice, K. (2006). FDR and B ayesian multiple comparisons rules. In Proc. Valencia/ISBA 8th World Meeting on Bayesian Statistics, Benidorm (Alicante, Spain)

    work page 2006

  31. [31]

    A., Noueiry, A., Sarkar, D., and Ahlquist, P

    Newton, M. A., Noueiry, A., Sarkar, D., and Ahlquist, P. (2004). Detecting differential gene expression with a semiparametric hierarchical mixture method . Biostatistics , 5(2):155--176

    work page 2004

  32. [32]

    and Casella, G

    Park, T. and Casella, G. (2008). The B ayesian lasso. Journal of the American Statistical Association , 103(482):681--686

    work page 2008

  33. [33]

    B., Stingo, F

    Peterson, C. B., Stingo, F. C., and Vannucci, M. (2016). Joint B ayesian variable and graph selection for regression models with network-structured predictors. Statistics in Medicine , 35(7):1017--1031

    work page 2016

  34. [34]

    G., Scott, J

    Polson, N. G., Scott, J. G., and Windle, J. (2013). B ayesian inference for logistic models using P \'o lya-- G amma latent variables. Journal of the American Statistical Association , 108(504):1339--1349

    work page 2013

  35. [35]

    Ren, Z., Wei, Y., and Cand \`e s, E. (2023). Derandomizing knockoffs. Journal of the American Statistical Association , 118(542):948--958

    work page 2023

  36. [36]

    J., Bickel, P

    Rothman, A. J., Bickel, P. J., Levina, E., and Zhu, J. (2008). Sparse permutation invariant covariance estimation. Electronic Journal of Statistics , 2(none):494 -- 515. Publisher: Institute of Mathematical Statistics and Bernoulli Society

    work page 2008

  37. [37]

    Savitsky, T., Vannucci, M., and Sha, N. (2011). Variable selection for nonparametric G aussian process priors: Models and computational strategies. Statistical Science , 26(1):130--149

    work page 2011

  38. [38]

    Scott, J. G. and Berger, J. O. (2010). B ayes and empirical- B ayes multiplicity adjustment in the variable-selection problem. The Annals of Statistics , 38(5):2587--2619

    work page 2010

  39. [39]

    Sesia, M., Sabatti, C., and Candès, E. J. (2018). Gene hunting with hidden M arkov model knockoffs . Biometrika , 106(1):1--18

    work page 2018

  40. [40]

    G., and Vannucci, M

    Sha, N., Tadesse, M. G., and Vannucci, M. (2006). B ayesian variable selection for the analysis of microarray data with censored outcomes. Bioinformatics , 22(18):2262--2268

    work page 2006

  41. [41]

    G., Brown, P

    Sha, N., Vannucci, M., Tadesse, M. G., Brown, P. J., Dragoni, I., Davies, N., Roberts, T. C., Contestabile, A., Salmon, M., Buckley, C., et al. (2004). B ayesian variable selection in multinomial probit models to identify molecular signatures of disease stage. Biometrics , 60(3):812--819

    work page 2004

  42. [42]

    Smith, A. F. (1973). A general B ayesian linear model. Journal of the Royal Statistical Society: Series B (Methodological) , 35(1):67--75

    work page 1973

  43. [43]

    A., Kabalin, J

    Stamey, T. A., Kabalin, J. N., McNeal, J. E., Johnstone, I. M., Freiha, F., Redwine, E. A., and Yang, N. (1989). Prostate specific antigen in the diagnosis and treatment of adenocarcinoma of the prostate. ii. radical prostatectomy treated patients. The Journal of urology , 141(5):1076--1083

    work page 1989

  44. [44]

    Tadesse, M. G. and Vannucci, M., editors (2021). Handbook of B ayesian Variable Selection . Chapman and Hall/CRC

    work page 2021

  45. [45]

    Tanner, M. A. and Wong, W. H. (1987). The calculation of posterior distributions by data augmentation. Journal of the American Statistical Association , 82(398):528--540

    work page 1987

  46. [46]

    Tibshirani, R. (1996). Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B (Statistical Methodology) , 58(1):267--288

    work page 1996

  47. [47]

    Vannucci, M. (2021). Discrete spike-and-slab priors: Models and computational aspects. Handbook of Bayesian Variable Selection , pages 3--24

    work page 2021

  48. [48]

    Villasenor Alva, J. A. and Estrada, E. G. (2009). A generalization of S hapiro-- W ilk's test for multivariate normality. Communications in Statistics—Theory and Methods , 38(11):1870--1883

    work page 2009

  49. [49]

    Wang, H. (2015). Scaling It Up: Stochastic Search Structure Learning in Graphical Models . Bayesian Analysis , 10(2):351 -- 377

    work page 2015

  50. [50]

    Whittemore, A. S. (2007). A B ayesian false discovery rate for multiple testing. Journal of Applied Statistics , 34(1):1--9

    work page 2007

  51. [51]

    Xing, X., Zhao, Z., and Liu, J. S. (2023). Controlling false discovery rate using G aussian mirrors. Journal of the American Statistical Association , 118(541):222--241

    work page 2023

  52. [52]

    Yap, J. K. and Gauran, I. I. M. (2023). B ayesian variable selection using knockoffs with applications to genomics. Computational Statistics , 38(4):1771--1790

    work page 2023

  53. [53]

    Zhou, M., Li, L., Dunson, D., and Carin, L. (2012). Lognormal and gamma mixed negative binomial regression. In Proceedings of the International Conference on Machine Learning. International Conference on Machine Learning , pages 1343--1350. NIH Public Access

    work page 2012

  54. [54]

    and Hastie, T

    Zou, H. and Hastie, T. (2005). Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society: Series B (Statistical Methodology) , 67(2):301--320

    work page 2005