Pith. sign in

REVIEW 3 major objections 5 minor 2 references

Model-free Feature Screening and FDR Control with Knockoff Features

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Ranking features by projection correlation achieves sure screening with no model, and adding a knockoff threshold yields simultaneous FDR control and sure screening whenever the target FDR level is at least 1/s.

desk verdict PC-Screen is a solid model-free screening method with new concentration results, but the knockoff-based FDR theory doesn't actually cover the implemented top-d algorithm. read the letter →

arxiv 1908.06597 v3 pith:INHQACK6 submitted 2019-08-19 stat.ME stat.ML

Wanjun Liu , Yuan Ke , Jingyuan Liu , Runze Li This is my paper · ORCID
classification stat.MEstat.ML
keywords featurescreeningprojectioncorrelationmodel-freeknockofffeaturesfalsediscoveryratesurerankconsistencyultra-highdimensionaldata
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

PC-Screen ranks features by their projection correlation with the response and is claimed to achieve sure screening—every active feature is retained with probability tending to 1—and rank consistency under only a minimum-signal-strength condition, with no regression model, no sub-Gaussian assumption, and no limit on the response dimension. The paper also proposes PC-Knockoff, a two-step procedure that first screens to a moderate set and then uses knockoff features to choose a threshold with false discovery rate control. The main theoretical result is that if the nominal FDR level $\alpha$ is at least $1/s$, where $s$ is the number of active features, PC-Knockoff controls FDR and keeps all active features simultaneously with high probability; below $1/s$ there is a phase transition and no such guarantee. These results give practitioners a model-free screening tool that works for heavy-tailed, nonlinear, and multivariate response data, and a principled data-adaptive way to set the screening threshold instead of picking a conservative cutoff by hand.

What carries the argument

The argument is carried by two paired objects. The first is the squared projection correlation $\omega_k = \mathrm{PC}(X_k,\mathbf y)^2$, whose sample version (2.4)–(2.5) is a triple-sum average of arccosine angles, and for which Theorem 1 provides a moment-free, dimension-free exponential deviation bound; this is what makes sure screening and rank consistency valid without model or tail assumptions. The second is the knockoff contrast $\hat W_j = \widehat{PC}(X_j,Y)^2-\widehat{PC}(\tilde X_j,Y)^2$, whose sign for inactive features is symmetric—exactly fair coin flips conditioned on the absolute values (Lemma 1)—and the knockoff+ threshold $T_\alpha$ from (3.6). Lemma 1 turns the threshold-selection problem into a backward super-martingale, so the optional stopping theorem yields the FDR bound in Theorem 4, and the signal-separation argument in Theorem 5 gives the simultaneous sure-screening guarantee at $\alpha\ge 1/s$.

What would settle it

Run Algorithm 1 at $\alpha = 0.25$ with $s=10$ on the heavy-tailed mixture data of Model 4.c and check whether the empirical FDR exceeds $0.25$ as reported (0.254 in Table 4); or, separately, simulate Condition 1(a) with a known active set and check whether a top-$d$ screening step retains all active features with high probability—if it does not, the event $\mathcal{E}$ on which Theorems 4 and 5 condition is not established for Algorithm 1.

Watch

Extended reading notes

Core claim

The paper's central claim is that projection correlation—the dependence measure defined in (2.1)–(2.2) as an average over all unit projections of the squared covariance of indicator transforms—is the right engine for model-free screening in ultra-high dimensions. Theorem 1 establishes a non-asymptotic exponential concentration inequality for the empirical squared projection correlation with constants that do not depend on dimension or on any moment conditions. From it, Theorem 2 gives sure screening, $\Pr(\mathcal{A}\subseteq\hat{\mathcal{A}}(\delta))\geq 1-O(s\exp\{-c_4 n^{1-2\kappa}\})$, when the smallest active signal exceeds $2c_3 n^{-\kappa}$, and Theorem 3 gives rank consistency under a signal-gap condition. On the FDR side, the paper proves that the knockoff+ threshold (3.6) applied to $\hat W_j=\widehat{PC}(X_j,Y)^2-\widehat{PC}(\tilde X_j,Y)^2$ controls the false discovery rate conditionally on the screening event $\mathcal{E}$ (Theorem 4), and that for $\alpha\geq 1/s$ it still retains every active feature with probability $1-O(n_2\exp\{-c_4 n_2^{1-2\kappa}\})$ (Theorem 5(i)); for $\alpha<1/s$, the procedure either recovers the whole active set or returns an empty set, with no sure-screening guarantee.

Load-bearing premise

The dual premise that the knockoff features are exact (swap-exchangeable and conditionally independent of the response) and that the first-stage top-$d$ screen already contains every active feature is what makes both FDR control and sure screening true, and the implemented algorithm guarantees neither—the paper's own Model 4.c shows FDR inflation when second-order knockoffs fail to be exact.

Editorial extensions

If this is right

  • Screening can be applied before any model is chosen: the same guarantees cover linear, nonlinear, additive, quantile, Poisson, and multivariate-response data, so model specification is no longer a prerequisite for dimension reduction.
  • With exact knockoffs, FDR control and sure screening are compatible exactly when the target level is not below $1/s$; below that, the procedure exhibits a hard phase transition and cannot promise both.
  • The screening threshold no longer needs to be fixed conservatively: Algorithm 1's knockoff step sets the cutoff data-adaptively while bounding false discoveries.
  • Because the concentration inequality is dimension-free and moment-free, the theoretical error rates do not degrade as $p$ grows or as tails become heavier, unlike distance-correlation screening whose rate carries an extra $\eta$ term.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural testable extension is to build a diagnostic that checks swap exchangeability of the constructed second-order knockoffs on the second subsample; the paper's Model 4.c result (empirical FDR 0.254 at $\alpha = 0.25$) suggests the guarantee can degrade badly when that diagnostic fails.
  • The phase transition at $1/s$ can be inverted into a formal estimator of the active-set size $s$ by scanning $\alpha$ and finding the largest level that yields an empty selection, as the paper sketches informally; a future analysis could attach confidence intervals to that estimate.
  • Because the $W$-statistic cancels spurious marginal signals of inactive features, screening on $\hat W$ may tolerate strong marginal correlations between inactive and active features better than PC-Screen itself, potentially opening a path to factor-model or confounded settings—though the paper does not analyze that regime.
  • The sample-splitting scheme in Algorithm 1 suggests a general template: any marginal dependence measure with a dimension-free concentration inequality could replace projection correlation, and the FDR step would remain valid as long as the first-stage event holds, a direction the paper notes but does not develop.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes two procedures for ultra-high-dimensional feature screening. PC-Screen ranks features by the sample projection correlation between each feature (or feature vector) and a possibly multivariate response, and screens either by threshold or by taking the top d features. The paper proves non-asymptotic exponential concentration inequalities for the sample projection correlation (Theorem 1) and uses them to establish sure screening (Theorem 2) under a minimum-signal-strength condition and rank consistency (Theorem 3) under a uniform signal gap. The second procedure, PC-Knockoff, first applies PC-Screen on one subsample to reduce the dimension to a moderate d, then constructs second-order knockoff features on the remaining subsample and selects features by the knockoff+ thresholding rule based on differences of sample projection correlations. Theorems 4 and 5 claim conditional FDR control and joint sure-screening-plus-FDR-control properties when exact knockoffs are available, with a phase transition at alpha = 1/s. The paper also reports extensive simulations and a real supermarket data analysis.

Significance. If the theoretical claims are fully established, the paper would make a useful contribution: PC-Screen is genuinely model-free, robust to heavy-tailed errors and multivariate responses, and the non-asymptotic concentration inequalities for projection correlation are of independent interest. The knockoff-based threshold selection idea for model-free screening is appealing and the conditional-FDR proof in Appendix A.1 is coherent. The numerical comparisons are extensive and show clear advantages for PC-Screen in the settings considered. However, two load-bearing gaps currently prevent the central claims from being accepted as stated: the connection between the thresholding theorem and the top-d screening step is missing, and the implemented second-order knockoffs are not covered by the FDR/sure-screening theorems.

major comments (3)
  1. [Section 3.4, Algorithm 1, Remark 4] The screening step of Algorithm 1 selects the top d features, but the only screening guarantee invoked, Theorem 2 (Eq. 2.6), is a thresholding result for the set {k: \hat\omega_k >= delta} and does not control the cardinality of that set or the event that the d-th largest sample correlation satisfies \hat\omega_{(d)} <= c3 n_1^{-\kappa}. Under Condition 1(a) alone, many inactive features can have population projection correlations as large as or larger than the active ones, so the top-d set may miss active features even when the thresholded set contains them. Consequently Pr(E), the event that all active features are in the top-d set, is not shown to tend to 1 under the stated assumptions. This undermines the unconditional statement in Remark 5 and the claim that Theorems 4 and 5 apply to Algorithm 1 as written; the authors should either impose Condition 1(b) or another explicit gap condition on the order statistics in Theorems 4 and 5, or replace the top-d rule by a threshold rule with a size guarantee, or directly bound Pr(\hat\omega_{(d)} <= c3 n_1^{-\kappa}).
  2. [Section 3.2-3.4, Theorems 4 and 5] The FDR and sure-screening guarantees are proved only for exact knockoff features satisfying Condition 2, but Algorithm 1 constructs second-order knockoffs from an estimated covariance matrix via (3.1)-(3.3). The paper acknowledges in Remark 3 and in the discussion of Table 4 (Model 4.c) that these approximate knockoffs may not be close to exact ones, and indeed the reported empirical FDR in Model 4.c at alpha = 0.25 is 0.254, exceeding the nominal level. No theoretical result bounds the FDR inflation or the loss of screening power caused by the second-order approximation. Thus the abstract's blanket statement that the proposed two-step approach controls FDR is not established for the implemented procedure; the authors should state the guarantees only for exact knockoffs, or provide explicit conditions and a bound on the approximation error that yields an FDR correction.
  3. [Section 3.4, Remark 5 and the paragraph before Theorem 4] The probability bound on the screening step is stated as 1 - O(s exp{c4 n1^{1-2kappa}}) and is described as following from Theorem 2, but the displayed expression has a sign error (the exponent should be -c4 n1^{1-2kappa}) and, more importantly, the bound holds only on the event that \hat\omega_{(d)} <= c3 n1^{-\kappa}. The probability of this order-statistic event is not bounded under the assumptions of Theorems 4 and 5, so the combined probability bound in Remark 5 does not follow as written. This is the same root gap as the previous comment, but it directly affects the stated rate in the main text and should be corrected explicitly.
minor comments (5)
  1. [Section 4.2, Model 4.c] The definition 'x = 0.9x1 + 0.1x2' is ambiguous about whether x is a scalar or a vector and what covariance matrix the t2 component has; please clarify the data-generating scheme.
  2. [Figure 1 caption] The word 'receptively' should be 'respectively'.
  3. [Section 4.1, Example 2] The sentence 'DC-SIS and bcDC-SIS preform comparably' contains a typo; 'preform' should be 'perform'.
  4. [Section 3.4, sentence before Theorem 4] The expression '1-O(s exp{c4 n1^{1-2kappa}})' is missing a minus sign inside the exponent; it should read '1-O(s exp{-c4 n1^{1-2kappa}})'.
  5. [Algorithm 1 and Eq. (3.7)] Algorithm 1 defines \hat A(T_alpha) as {j: j in \hat A1, \hat W_j >= T_alpha}, while Eq. (3.7) defines \hat A(T_alpha) over all p features; the notation should be reconciled so that the final selected set is unambiguously the intersection with \hat A1.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: central theorems follow from new concentration bounds and knockoff exchangeability; self-citations are background.

full rationale

The paper contains no circular step. PC-Screen and PC-Knockoff are derived from a new exponential deviation inequality for the sample projection covariance (Theorem 1), proved in the supplement from U-statistic concentration, and from the standard Model-X knockoff exchangeability Lemma 1, which follows from Condition 2 (exact knockoffs) exactly as in Barber and Candes; no parameter is fitted and then renamed as a prediction, and no target theorem is assumed as an input. Self-citations to Zhu et al. (2017) supply the definition and U-statistic formula for projection correlation; these are external, parameter-free results with stated assumptions and do not include the paper's screening or FDR conclusions, so under the review rules they are legitimate support rather than load-bearing circularity (see Section 2.1, equations (2.1)-(2.3)). The paper's own acknowledged limitations — Remark 3 on second-order knockoffs, Table 4 Model 4.c FDR = 0.254 at alpha = 0.25, and the Section 3.4/Remark 4 reliance on the unproved condition \hat{\omega}_{(d)} <= c3 n1^{-\kappa} to make the top-d screening event E hold — are correctness or assumption-coverage issues, not reductions of the conclusions to their inputs. Theorems 2-6 are stated under explicit Conditions 1-2 and proved from concentration and exchangeability; the top-d screening gap does not make any result definitionally equivalent to an input. The empirical sections compare against external benchmarks, further confirming that the central claims are not internally manufactured. Score 0.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The main guarantees are conditional on standard screening assumptions (minimum signal strength, non-degeneracy), exact knockoffs, and an event E for top-d screening that is not proved for the algorithm as implemented. The user-chosen hyperparameters d, n_1, and threshold delta are part of the procedure's practical input.

free parameters (3)
  • Screening threshold delta = c_3 n^{-kappa} with unspecified c_3 and kappa
    PC-Screen selects the set {k: \hat omega_k >= delta}; the theory requires delta <= min active omega / 2 and suggests delta = c_3 n^{-kappa}, but c_3 and kappa are not identified from data. PC-Knockoff is proposed to avoid this choice.
  • Screening target dimension d = 100 in the simulation study; user-specified
    Algorithm 1 requires d < n_2/2; the theory is conditional on d being large enough that the top d features contain all actives, but no principled choice is given.
  • First-subsample size n_1 = 250 in the simulation study; user-specified
    Algorithm 1 splits the sample into n_1 and n_2; the authors suggest a small n_1 but provide no quantitative guidance.
assumptions (6)
  • domain assumption Projection correlation properties from Zhu et al. (2017), including PC=0 iff independence for jointly continuous random vectors and the U-statistic representation of Pcov.
    Section 2.1 adopts these results without proof; the screening and knockoff statistics are built on them.
  • domain assumption Non-degeneracy: min over k of Pcov(X_k,X_k)^2 >= sigma_0^2 and min over l of Pcov(Y_l,Y_l)^2 >= sigma_0^2.
    Section 2.2 states this to keep population projection correlations well-defined and bounded away from zero.
  • domain assumption Minimum signal strength Condition 1(a): min over active k of omega_k >= 2 c_3 n^{-kappa}; Condition 1(b): min active omega - max inactive omega >= 2 c_3 n^{-kappa}.
    Section 2.2, Conditions 1(a) and 1(b), are needed for Theorem 2 (sure screening) and Theorem 3 (rank consistency).
  • domain assumption Exact knockoff Condition 2: swap exchangeability of (x, \tilde x) and \tilde x independent of y given x.
    Section 3.2, Condition 2; Lemma 1 and Theorems 4-6 rely on exact knockoffs, while Algorithm 1 implements approximate second-order knockoffs.
  • domain assumption Signal strength on W: min over active k of W_k >= 4 c_3 n_2^{-kappa} in Theorem 5.
    Section 3.4, Theorem 5; needed for the active/inactive separation on the knockoff scale.
  • domain assumption Joint continuity of (X_k, y) for the property that PC=0 iff independence holds.
    Section 2.1 notes the equivalence can fail for discrete variables; several simulations use discrete or count responses (Models 1.f, 4.e, supermarket data), so the screening theory does not strictly cover these settings.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Model-free Feature Screening and FDR Control with Knockoff Features." pith.science (2026). https://pith.science/paper/INHQACK6

@misc{pith2026190806597,
  author       = {Pith},
  title        = {Pith review of: Model-free Feature Screening and FDR Control with Knockoff Features},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/INHQACK6}},
  note         = {Machine review of arXiv:1908.06597}
}
abstract

This paper proposes a model-free and data-adaptive feature screening method for ultra-high dimensional datasets. The proposed method is based on the projection correlation which measures the dependence between two random vectors. This projection correlation based method does not require specifying a regression model and applies to the data in the presence of heavy-tailed errors and multivariate response. It enjoys both sure screening and rank consistency properties under weak assumptions. Further, a two-step approach is proposed to control the false discovery rate (FDR) in feature screening with the help of knockoff features. It can be shown that the proposed two-step approach enjoys both sure screening and FDR control if the pre-specified FDR level $\alpha$ is greater or equal to $1/s$, where $s$ is the number of active features. The superior empirical performance of the proposed methods is justified by various numerical experiments and real data applications.

Figures

Figures reproduced from arXiv: 1908.06597 by the authors.

Figure 1
Figure 1. Scatter plots between the response and variables [PITH_FULL_IMAGE:figures/full_fig_p032_1.png] view at source ↗
Figure 2
Figure 2. Scatter plots between the response and variables [PITH_FULL_IMAGE:figures/full_fig_p033_2.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

2 extracted references · 2 canonical work pages

  1. [1]

    Barber, R. F. and Cand` es, E. J. (2015), ‘Controlling the false discovery rate via knockoffs’, The Annals of Statistics 43(5), 2055–2085. Barber, R. F. and Cand` es, E. J. (2019), ‘A knockoff filter for high-dimensional selective inference’, The Annals of Statistics 47(5), 2504–2537. Cand` es, E., Fan, Y., Janson, L. and Lv, J. (2018), ‘Panning for gold: ‘M...

    work page 2015

  2. [911]

    and Wu, Y

    Fan, J., Samworth, R. and Wu, Y. (2009), ‘Ultrahigh dimensional feature selection: beyond the linear model’, Journal of Machine Learning Research 10(9), 2013–2038. Fan, J. and Song, R. (2010), ‘Sure independence screening in generalized linear models with NP-dimensionality’, The Annals of Statistics 38(6), 3567–3604. Fan, Y., Demirkaya, E., Li, G. and Lv,...

    work page 2009

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.