{"id":"aee5bf25-9f3c-4e15-988a-37b94dc7ae63","arxiv_id":"2508.18901","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A penalized continuous mRMR objective with SCAD/MCP penalties and a knockoff filter is proposed for FDR-controlled feature screening, with sparsistency theory and experiments.","lead":"The paper proposes SmRMR, a feature screening method that combines feature-target relevance with feature-feature redundancy in a penalized optimization, then wraps it in a knockoff filter to claim false-discovery control. It is a close variant of the existing HSIC-LASSO with SCAD/MCP penalties, and its empirical benefit is a smaller selected feature set at some accuracy cost.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper proves sparsistency for the pseudo-true mRMR parameter θn0, but never proves that its zero entries equal the conditional inactive set I; the abstract's central claim rests on this unproven identification step.","rationale":"The reader's verdict is CONDITIONAL with moderate confidence. I agree with the conditional verdict, but I identify the identification gap as the single most load-bearing concern rather than Assumption 3.2. Assumption 3.2 is a technical eigenvalue restriction that limits applicability to strongly dependent features, and it is a real concern; however, the paper's core conceptual claim—that zero coefficients identify inactive features—would still be unsupported even if Assumption 3.2 held. The theorems as stated recover the support of θn0, a pseudo-true parameter defined by a quadratic loss over marginal dependence measures. There is no theorem connecting support(θn0) to the conditional-independence set I. The experiments evaluate against causal features, so the evaluation relies on exactly this unproven equivalence. This is not a question of consensus versus non-consensus; it is an internal gap between the definitions in Section 2.1, the estimator in Section 2.3, and the theorems in Sections 3 and 4. A concrete simulation or analytic derivation can settle whether the equivalence happens to hold in important cases; if it fails, the abstract and Section 1 need re-framing, and the FDR claim needs to be stated for the mRMR-active set rather than for causal inactive features. Therefore the reader's CONDITIONAL verdict stands, with the condition made more precise: prove or drop the identification of θ0 zeros with I, and report FDR relative to the appropriate target set.","tokens_in":29134,"tokens_out":6148,"duration_ms":63938,"concrete_test":"Generate data from Y = X1 + ε with X1 ~ N(0,1), X2 = X1 + δ (δ small noise), so X2 is inactive conditional on X1 but D(X2,Y) > 0. Estimate D with HSIC (Gaussian kernel) and with PC on a very large sample (or use exact population values when available), then solve the population problem min_{θ≥0} L(θ) for D. Check whether support(θ0) = {1}. Repeat with nonlinear Y = f(X1)+ε and with an additional pure-noise feature X3. If support(θ0) ever contains a conditionally inactive feature or omits an active one, the identification claim fails; if support matches I in all tested cases, the missing proof may be supplyable.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The abstract and Section 1 claim that zero coefficients identify inactive features, where Section 2.1 defines I by conditional independence: F(Y|X) does not depend on X_k. What is actually proved is different. Theorem 3.6 shows that, under assumptions, the zero coordinates of the pseudo-true parameter θn0 are estimated as zero with probability tending to one. But θn0 is defined in Assumption 3.1 only as the minimizer of L(θ) = -Σ_k θ_k D(X_k,Y) + (1/2)Σ_{k,l} θ_k θ_l D(X_k,X_l) over θ ≥ 0. No proposition establishes {k:θn0,k=0} = I. This is not a trivial gap: D(X_k,Y) is a marginal dependence measure (HSIC or PC), so a conditionally inactive feature can be marginally dependent on Y through correlation with active features. For the quadratic criterion, the stationarity condition is only D_XX θ0 = D_XY when D is used on both sides; for general nonparametric dependence measures, the support of θ0 has no established relation to conditional independence. Thus Theorem 3.6's 'non-relevant features' are non-relevant for the mRMR loss, not necessarily the inactive set I named in the abstract. The FDR guarantee in Theorem 4.1 is likewise relative to Sn={k:θn0,k≠0}, not to the causal inactive features used in the experiments' FDR/TPR evaluation. Section 2.3's heuristic only shows that a zero θk implies a bound involving bDv(X_k,Y) and redundancy terms, which is not conditional independence.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a sparse continuous relaxation of mRMR feature screening, called SmRMR, in which a penalty (Lasso, SCAD, or MCP) is applied to a quadratic loss built from pairwise dependence estimates (HSIC or projection correlation). The main theoretical claims are: (i) consistency of a local minimizer of the penalized criterion (Thm 3.5), (ii) sparsistency, i.e. recovery of zero coefficients of the population pseudo-true parameter (Thm 3.6), and (iii) conditional FDR control after a two-stage knockoff procedure (Thm 4.1). Experiments on synthetic and real datasets compare SmRMR with HSIC-LASSO and report accuracy, TPR, FDR, and number of selected features. Code is provided.","tokens_in":29608,"tokens_out":8796,"duration_ms":95363,"significance":"If the claims were fully established, the paper would contribute a redundancy-aware screening method with an FDR guarantee, extending HSIC-LASSO in a useful direction. The authors provide reproducible code, a wide experimental study, and proofs that follow the standard Fan-Li / Loh-Wainwright template for penalized M-estimation. The most valuable and defensible part is the support-recovery analysis for the pseudo-true mRMR parameter and the conditional FDR statement for a valid knockoff construction. However, the paper's central advertised claim — that zero coefficients identify the conditional-independence inactive set of Section 2.1 — is never proved, and several assumptions are in tension with the method's motivation. These issues are load-bearing for the paper as written.","major_comments":[{"comment":"The paper claims that zero coefficients in the estimated parameter correspond to the inactive features I = {k : F(Y|X) does not functionally depend on X_k}. What is actually proved is that the penalized estimator recovers the support of the pseudo-true parameter θ_n0 defined in Assumption 3.1 as the minimizer of L(θ) = -Σθ_k D(X_k,Y) + (1/2)Σθ_k θ_l D(X_k,X_l). No proposition establishes {k : θ_n0,k=0} = I. Since D is a marginal dependence measure, a conditionally inactive feature can be marginally dependent on Y through active features, and the redundancy term can drive its coefficient to zero even when it is conditionally active. The heuristic in Section 2.3 only gives inequalities involving bD_v(X_k,Y) and redundancy terms, not conditional independence. Consequently Theorem 3.6 concerns 'non-relevant' features for the mRMR loss, not the inactive set named in the abstract; the FDR theo","section":"Sections 1, 2.3, 3 (Thms 3.5–3.6)"},{"comment":"Assumption 3.1 states that an open neighbourhood of θ_n0 is contained in R_+^{p_n} and that the stationarity equality -D(X_k,Y)+Σ_l D(X_k,X_l)θ_n0,l = 0 holds for every k. If θ_n0 has zero coordinates (as required for sparsistency and as assumed by the sparsity structure before Theorem 3.5), no Euclidean open neighbourhood of θ_n0 is contained in the non-negative orthant. The correct first-order condition at a boundary zero is a variational inequality / directional derivative condition, not equality. The proof of Theorem 3.6 uses the equality form of Assumption 3.1 for zero coordinates, so the current assumptions are internally inconsistent with the very sparsity pattern the theorems are meant to establish. This needs to be repaired, e.g. by formulating boundary KKT conditions and adapting the proofs.","section":"Assumption 3.1, Section 3"},{"comment":"Assumption 3.2 requires 0 < μ ≤ λ_min(D_XX) ≤ λ_max(D_XX) < μ < ∞. This excludes situations where two features are perfectly or nearly redundant, since then D_XX is singular or ill-conditioned. But redundancy among features is the exact motivation of mRMR, and the paper's own DGPs include correlated features (e.g. DGPs 1.c, 3.c with c=0.5). Moreover, Theorem 3.5 requires p_n^2 s_n log(p_n) = o(n), which forces p_n ≪ sqrt(n) when s_n ≥ 1. This is incompatible with the paper's stated ultra-high-dimensional setting (p ≫ n) and with experiments such as n=100, p=5000. The theory should either be restricted to moderate-dimensional problems or extended with screening theory and restricted-eigenvalue-type conditions; in its present form, the main theorems do not cover the regime the method is designed for.","section":"Assumption 3.2, Theorems 3.5–3.6, Section 1"},{"comment":"The FDR guarantee in Theorem 4.1 relies on the knockoff exchangeability property (via Lemma 1 of Liu et al. [2022]). But the implementation in Section 5.1.2 uses second-order Model-X knockoffs with the equicorrelated construction. For non-Gaussian X and for kernel-based statistics that depend on the full joint distribution, second-order knockoffs are not exact knockoffs, so the sign-flip property required for W_k may fail. Thus the theorem does not justify the FDR numbers reported in the experiments unless exact Model-X knockoffs or an appropriate second-order-sufficient statistic is established. In addition, Theorem 4.1 is conditional on the screening event E_n = {S_n ⊆ hat S_0,n, 2s_0,n < n_1}; the experiments evaluate FDR with respect to the causal active set S, not S_n = {k : θ_n0,k ≠ 0}. The authors should make this distinction explicit and, if the causal claim is retained, connect","section":"Section 4, Section 5.1.2, Theorem 4.1"},{"comment":"The paper acknowledges that the modified procedure SmRMR2, which returns at least one feature, has FDR not controlled in every situation. This is a clear and honest statement, but it means the headline FDR control in the abstract and in parts of the experimental section applies only to the unmodified procedure; the figures and tables mix both variants. The authors should separate the two regimes and clearly label where the theoretical guarantee applies, especially because SmRMR2 is used for real-data benchmarking.","section":"Section 5.3, SmRMR2"}],"minor_comments":[{"comment":"Typo: 'righ-derivative' should be 'right-derivative'. The subdifferential of the LASSO at θ_k=0 under the non-negative constraint is stated as {z_k : z_k ≤ 1}; for the non-negative constrained problem the relevant subgradient interval should also reflect the sign constraint (z_k ∈ [0,1] when θ_k=0 in the one-sided sense). Please clarify.","section":"Section 2.3"},{"comment":"'Bayes optimization' should be 'Bayesian optimization'. Also 'fine-tuning the LARS penalty' is unclear; LARS does not normally have a penalty tuning parameter.","section":"Section 5.1.2"},{"comment":"The stopping criterion 'while s ≠ m and ||bθ(s) − bθ(s−1)|| < ε' combines a fixed iteration cap with a convergence check; as written, if s reaches m the loop exits regardless of convergence. State explicitly whether convergence is required before m iterations.","section":"Algorithm 1"},{"comment":"The objective G_{v,n} includes cross terms only between original variables and only between knockoff variables, but no cross term X_k with tilde X_l. This is a modelling choice; a sentence explaining why the cross term is omitted would help, since a fully quadratic model would normally include it.","section":"Section 4, Eq. (4.1)"},{"comment":"Lemma A.2's proof invokes 'Lemma S.2 of Liu et al. [2022]' without stating the lemma or its conditions. Since the paper otherwise provides self-contained proofs, please state the lemma or give a direct argument for the normalized HSIC deviation.","section":"Appendix A"}],"recommendation":"major_revision","confidential_remarks":"The identification gap is the most serious issue: it is not a matter of a missing line in a proof, because the equivalence between zero θ_n0 coefficients and conditional-independence inactivity is generally false for marginal dependence measures. The authors should be firmly pushed to either prove an identification theorem under explicit conditions or substantially reframe the contribution as support recovery for the pseudo-true mRMR parameter, with all abstract/conclusion/experimental FDR claims adjusted accordingly. I do not recommend rejection outright because the revised framing could still be publishable, but the revision is substantial."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid engineering-plus-theory paper, but the main selling point as written overreaches. The abstract says zero coefficients represent the inactive features defined by conditional independence in Section 2.1. What the theorems actually prove is different: theta_0 is defined as the minimizer of the mRMR loss L(theta) = -sum theta_k D(X_k,Y) + (1/2) sum theta_k theta_l D(X_k,X_l), and Theorems 3.5 and 3.6 show that the penalized estimator is consistent and sparsistent for the support of that theta_0. No proposition connects {k: theta_0,k = 0} to the conditional independence set I. That gap is not cosmetic. D(X_k,Y) is a marginal dependence measure, so a conditionally inactive feature can be marginally dependent on Y through other features. The Section 2.3 heuristic only gives inequalities involving redundancy terms; it does not establish conditional independence. This is the stress-test point, and I think it holds up.\n\nWhat is genuinely new: the SCAD/MCP-penalized mRMR objective, the projection-correlation variant, the sparsistency results for the pseudo-true parameter, and the knockoff screening pipeline. The experiments are more honest than many: the paper explicitly reports that SmRMR2, the variant that returns at least one feature, does not control FDR in every situation, and it acknowledges that HSIC-LASSO is usually more accurate. The practical benefit claimed - fewer selected features while keeping comparable accuracy - is visible in the figures.\n\nSoft spots, in proportion. The identification gap is the big one and it is load-bearing for the abstract's claim. The FDR theorem is conditional on the screening event E_n and is relative to S_n = {k: theta_0,k != 0}, not to the causal inactive set used in the experiments' TPR/FDR evaluation. The SmRMR2 variant used in the benchmark explicitly breaks FDR, which limits the practical FDR conclusions. Also, Assumption 3.2 requires a uniform lower bound on the eigenvalues of D_XX, which excludes the strongly correlated features that motivate mRMR in the first place; the rate condition p^2 s_n log p = o(n) also does not cover p=5000, n=100. These are real limitations, but they do not make the core math wrong - just narrower than the paper's framing.\n\nThe proofs look like a competent application of the Fan-Li template, and the knockoff part is standard. The citation pattern is unobjectionable; HSIC-LASSO and the knockoff literature are properly cited.\n\nWho is this for? People working on model-free feature screening who want a single-threshold, redundancy-aware alternative to HSIC-LASSO. It deserves a serious referee: the idea is sensible, there is reproducible code, and the theoretical gaps are fixable by re-framing. I would recommend peer review, but with the expectation of major revision: drop or prove the conditional-independence identification, and report unconditional FDR or screening failure rates instead of only the conditional guarantee.","headline":"Competent HSIC-LASSO extension with SCAD/MCP and knockoffs, but the headline claim that zero coefficients identify conditionally inactive features is not proven; the theorems only recover the support of the mRMR loss minimizer.","tokens_in":935,"tokens_out":1052,"would_cite":false,"duration_ms":28685,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F12","62H20","62J07"],"pacs":[],"model":"deepseek-v4-flash","headline":"A penalized mRMR criterion can mark inactive features by zero coefficients and control false discoveries with a knockoff filter.","keywords":["feature screening","mRMR","non-convex penalization","false discovery rate","knockoff filter","sparsistency","HSIC","projection correlation"],"falsifier":"Simulate a design with a duplicated or near-duplicated feature, e.g. X_2 = X_1 exactly while Y depends only on X_1, and run SmRMR with SCAD and MCP at large n. Since D_XX is singular or nearly singular, Assumption 3.2 fails; recording whether the redundant coefficient is consistently set to zero and whether the knockoff threshold still controls FDR across seeds would reveal exactly where the guarantee breaks.","tokens_in":29069,"feed_emoji":"🎯","tokens_out":5557,"duration_ms":57051,"temperature":0.7,"pith_summary":"This paper tries to establish that feature screening can be made redundancy-aware and FDR-controlled at the same time. It proposes a continuous, penalized version of the minimum Redundancy Maximum Relevance criterion, where the estimated coefficient for each feature is positive if the feature is relevant and zero if it is inactive. The paper proves that, with non-convex SCAD or MCP penalties, the zero pattern of the population mRMR solution is recovered consistently, and that adding a knockoff filter after a screening step controls the false discovery rate. If correct, the practical payoff is a feature selection method where the user sets a target FDR instead of guessing how many features to keep.","feed_headline":"Zeroed-out coefficients mark inactive features in sparse mRMR","feed_subtitle":"A penalized mRMR plus knockoff filter finds inactive features and controls false discoveries—no preset feature count.","key_machinery":"The engine is the penalized mRMR loss L_v,n(θ), a continuous relaxation of the discrete mRMR selection problem, regularized by a non-convex penalty such as SCAD or MCP. The feature-feature dependence matrix D_XX enters as a design-like quadratic form, and the association measures are V-statistic estimators—normalized HSIC and squared projection correlation—that make the loss an explicit least-squares or tensor least-squares problem. A data-splitting knockoff procedure supplies the FDR control, with the importance score defined as the difference between the fitted coefficient of an original feature and its knockoff copy.","core_discovery":"The central claim is that the sparse mRMR estimator, defined as the penalized continuous relaxation of the classic mRMR objective, consistently recovers the support of the population mRMR coefficients, so zero entries correctly identify the inactive features. Theorems 3.5 and 3.6 establish consistency and sparsistency under SCAD or MCP penalties, with the LASSO failing the sparsistency rate condition. Theorem 4.1 then shows that, conditional on a screening event where all truly active features survive the first data-splitting stage, the knockoff+ threshold controls the FDR of the selected set. The method can be built with either normalized HSIC or squared projection correlation as the depend","pith_inferences":["I read the theory as implying that the screening power comes from the feature-feature quadratic term: when two features are strongly redundant, the penalty can set one to zero even if both are marginally relevant, something marginal screening alone cannot do.","The paper asserts rather than proves that zero entries of the population mRMR coefficients coincide with the causal inactive set I of Section 2.1; the theorems show the estimator recovers the zero pattern of the mRMR solution, not that this pattern equals conditional independence with Y.","A natural extension is to apply the same screening-plus-knockoff structure with other dependence measures or with adaptive weights, since the proofs only need V-statistic estimators with the same sub-exponential deviation tail.","The eigenvalue bound on D_XX suggests the guarantee is weakest exactly in the regime of perfect or near-perfect feature redundancy, which is the regime mRMR was originally designed to address; testing with duplicated or near-duplicated features would map where the theory stops and the heuristics begin."],"forward_implications":["Under SCAD or MCP penalties, the zero coefficients of the SmRMR estimator can be trusted, with probability tending to one, to mark the features that the population mRMR criterion treats as inactive.","After screening, users can select features by setting an FDR level rather than by fixing the number of features to retain, which removes a common calibration step in feature screening pipelines.","Because the loss is a least-squares or tensor least-squares problem, the method can reuse standard solvers for LASSO, SCAD, and MCP, including local linear approximation algorithms.","Empirically, SmRMR reaches accuracy comparable to HSIC-LASSO while selecting substantially fewer features, which lowers the observed FDR in the reported simulations and real-data benchmarks."],"supporting_citations":[{"why":"Supplies the original mRMR criterion and the redundancy problem that the penalized relaxation is designed to solve.","marker":"Peng et al. [2005]"},{"why":"Provides the SCAD penalty and the oracle-style consistency machinery that Theorem 3.5 and 3.6 extend to the SmRMR loss.","marker":"Fan and Li [2001]"},{"why":"Supplies the MCP penalty, the other non-convex regularizer whose sparsistency is proved.","marker":"Zhang [2010]"},{"why":"Establishes the knockoff filter and the knockoff+ threshold that Theorem 4.1 adapts to the SmRMR screening pipeline.","marker":"Barber and Candès [2015]"},{"why":"Provides the model-X second-order knockoff construction used to build knockoff variables in the empirical procedure.","marker":"Candès et al. [2018]"},{"why":"Supplies the deviation inequality for projection correlation and the model-free knockoff FDR lemma that the proofs invoke.","marker":"Liu et al. [2022]"},{"why":"Defines HSIC-LASSO, the baseline method that SmRMR extends and compares against.","marker":"Yamada et al. [2014]"},{"why":"Defines projection correlation and its V-statistic estimator, one of the two dependence measures used in the SmRMR loss.","marker":"Zhu et al. [2017]"},{"why":"Provides the U-statistic decomposition of the HSIC estimator that underlies the deviation inequality for normalized HSIC.","marker":"Gretton et al. [2007]"},{"why":"Supplies the data-splitting screening-plus-knockoff template and the empirical observation about empty-set returns that motivates the SmRMR2 variant.","marker":"Poignard et al. [2022]"}],"fun_headline_variants":["Sparse mRMR with knockoff filter flags inactive features","Zero coefficients reveal inactive features in sparse mRMR","Penalized mRMR zeros out inactive features, controls FDR","No preset feature count: sparse mRMR with knockoff controls FDR","Knockoff-filtered sparse mRMR spots inactive features, controls FDR"],"cache_read_input_tokens":2688,"weakest_assumption_plain":"The load-bearing premise is Assumption 3.2: the feature-feature dependence matrix D_XX must have all eigenvalues bounded away from zero and infinity, which fails when features are strongly or perfectly redundant—even though handling redundancy is the stated motivation of the method.","fun_headline_variants_meta":{"raw":{"variants":["Sparse mRMR with knockoff filter flags inactive features","Zero coefficients reveal inactive features in sparse mRMR","Penalized mRMR zeros out inactive features, controls FDR","No preset feature count: sparse mRMR with knockoff controls FDR","Knockoff-filtered sparse mRMR spots inactive features, controls FDR"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000914,"raw_usage":{"total_tokens":3745,"prompt_tokens":712,"completion_tokens":3033,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":456,"completion_tokens_details":{"reasoning_tokens":2954}},"tokens_in":456,"tokens_out":3033,"duration_ms":18816,"temperature":1.0,"reasoning_tokens":2954,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-05T16:07:06.415835+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate a design with a duplicated or near-duplicated feature, e.g. X_2 = X_1 exactly while Y depends only on X_1, and run SmRMR with SCAD and MCP at large n. Since D_XX is singular or nearly singular, Assumption 3.2 fails; recording whether the redundant coefficient is consistently set to zero and whether the knockoff threshold still controls FDR across seeds would reveal exactly where the guarantee breaks.","supporting_citations":[{"cited_title":"Feature selection based on mutual information: Criteria of max-dependency, max-relevance, and min-redundancy","cited_arxiv_id":null,"evidence_quote":"Supplies the original mRMR criterion and the redundancy problem that the penalized relaxation is designed to solve."},{"cited_title":"Variable selection via nonconcave penalized likelihood and its oracle properties","cited_arxiv_id":null,"evidence_quote":"Provides the SCAD penalty and the oracle-style consistency machinery that Theorem 3.5 and 3.6 extend to the SmRMR loss."},{"cited_title":"Xing, and Masashi Sugiyama","cited_arxiv_id":null,"evidence_quote":"Defines HSIC-LASSO, the baseline method that SmRMR extends and compares against."},{"cited_title":"Projection correlation between two random vectors","cited_arxiv_id":null,"evidence_quote":"Defines projection correlation and its V-statistic estimator, one of the two dependence measures used in the SmRMR loss."},{"cited_title":"Teo, Le Song, Bernhard Sch \\\"o lkopf, and Alex Smola","cited_arxiv_id":null,"evidence_quote":"Provides the U-statistic decomposition of the HSIC estimator that underlies the deviation inequality for normalized HSIC."},{"cited_title":"Fermanian","cited_arxiv_id":null,"evidence_quote":"Supplies the data-splitting screening-plus-knockoff template and the empirical observation about empty-set returns that motivates the SmRMR2 variant."}],"review_version":1}