Pith. sign in

REVIEW 3 major objections 4 minor 1 cited by

ROOFS: RObust biOmarker Feature Selection

T0 review · 3 major / 4 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that, on the PIONeeR lung-cancer dataset (435 patients, 214 features after multicollinearity reduction), a statistical filter combining Benjamini-Hochberg-adjusted p-values from a t-test and logistic regression selects the

desk verdict Genuine software contribution, but the headline empirical claim is undercut by a circular semi-synthetic design. read the letter →

arxiv 2601.05151 v3 pith:MLFL2YFX submitted 2026-01-08 stat.ML cs.LG

Anastasiia Bakhmach , Paul Dufossé , Simon Charpigny , Florence Monville , Laurent Greillier , Fabrice Barlési , Sébastien Benzekry This is my paper · ORCID
classification stat.MLcs.LG
keywords featureselectionbiomarkerdiscoverystabilityoptimismcorrectionsemi-syntheticbenchmarkBenjamini-HochbergLASSOimmunotherapyresistance
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces ROOFS, a Python package that benchmarks many feature-selection (FS) methods on a user's dataset and reports stability, predictive performance with optimism correction, and true/false discovery rates from a semi-synthetic outcome. Applying it to PIONeeR clinical trial data on resistance to anti-PD-(L)1 immunotherapy, the authors compare 23 FS methods combined with 11 classifiers. They find that a simple filter—the union of Benjamini-Hochberg-adjusted p-values from a t-test and logistic regression at a 0.05 threshold—achieves the best balance of optimism-corrected AUC (0.72), stability (0.39), and true-positive rate (0.93) among all methods, including widely used LASSO (AUC 0.69, stability 0.34). If correct, this means simple statistical filters can be more stable and at least as predictive as popular embedded machine-learning methods in the common biomedical setting where the number of predictors is close to the number of patients, and that benchmarking FS methods before choosing one is worth the computational cost.

What carries the argument

The central object is the ROOFS benchmarking pipeline, which evaluates FS methods via bootstrap resampling (B = 100): identical preprocessing, FS, and classification are applied to each resample. Predictive performance is estimated with the .632+ optimism-correction method; stability is measured with Nogueira's frequency-based index; and discovery power is assessed on a semi-synthetic dataset where a simulated outcome is generated from 19 pre-selected real features, allowing true-positive and false-positive rates to be computed over 40 noise replicates. The winning method itself is a simple filter: it takes the union of features selected by BH-adjusted p-values (0.05 threshold) from a t-test

What would settle it

Re-run the semi-synthetic experiment with the 19 true predictors chosen by an independent rule (randomly selected features, features chosen by LASSO on the real outcome, or a nonlinear generative model with interactions) while keeping all other benchmark settings fixed. If p.adjust's true-positive rate no longer exceeds that of Fisher score or t-score, or if a different method wins, the original ranking is an artifact of the ground-truth construction.

Watch

Extended reading notes

Core claim

On the post-VIF PIONeeR dataset, the p.adjust filter—the union of features with Benjamini-Hochberg-adjusted p-values below 0.05 from a two-sample t-test and from logistic regression adjusted for PD-L1—was identified as the optimal FS method. It achieved the highest optimism-corrected AUC (0.72) when paired with a gradient boosting classifier, a stability of S = 0.39 by Nogueira's measure, and on the semi-synthetic benchmark a true-positive rate of 0.93 ± 0.06 (with a false-positive rate of 0.10 ± 0.03). This outperformed the LASSO baseline (AUC 0.69, S = 0.34) and all other tested methods, including resampling-based ensembles designed to improve stability (Bolasso, stability selection, Stabl

Load-bearing premise

The benchmark's simulated 'true' predictors were themselves chosen by Benjamini-Hochberg-adjusted p-values from a Cox model on the same dataset—the same family of procedure as the winning p.adjust filter—so if that ground-truth choice is not independent, the claim that p.adjust is the best discovery method on this data is not established.

Editorial extensions

If this is right

  • If the central claim holds, applied researchers in n≈p biomedical settings can safely start with simple statistical filters, which are computationally cheap, stable, and predictive, rather than defaulting to LASSO or other embedded methods.
  • The finding that resampling-based ensembles (Bolasso, stability selection, Stabl, RENT) did not beat plain LASSO in either stability or AUC suggests their added complexity may not pay off in this type of data.
  • The p.adjust signature of 22 features achieved the same optimism-corrected AUC as the full 214-feature model (0.72 vs 0.71), indicating that a parsimonious signature can match full-data prediction in this setting.
  • The high true-positive rate (0.93) but also high false-positive rate (0.10) for p.adjust implies a discovery-versus-error trade-off: for hypothesis generation, tolerating false positives may be acceptable, but for strict biomarker panels other methods may be preferable.
  • Because stability and AUC are only weakly correlated (Spearman r = 0.32), the paper implies that evaluating both is necessary before choosing an FS method, otherwise a stable but weak method (e.g., CIFE) or an unstable but strong one could be mistakenly preferred.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The semi-synthetic ground truth in this benchmark was constructed by selecting 19 'true' predictors using a 0.01 threshold on Benjamini-Hochberg-adjusted p-values from a Cox model fit to the same PIONeeR data—essentially the same family of p-value-based selection logic as the winning p.adjust filter. If this construction predisposes the simulation toward linear, p-value-detectable signals, the TPR
  • A direct extension would be to rerun the ROOFS benchmark with the synthetic outcome generated from nonlinear or interaction terms (the paper states ROOFS supports polynomial and pairwise-interaction terms), which would test whether p.adjust's high TPR persists when the true model is not linear in the features.
  • The paper's report of a successful application to the separate SChISM study (nine-variable fragmentomics signature, AUC 0.89) suggests the pipeline is transferable; a systematic cross-dataset comparison could establish which FS method families are consistently competitive, beyond this single PIONeeR example.
  • The per-feature selection frequencies and robustness measures reported by ROOFS may turn out to be more practically valuable than the single 'optimal' method, because they let clinicians see which biomarkers are selected consistently across many methods and bootstraps; this hypothesis could be tested by externally validating the high-frequency features on independent immunotherapy cohorts.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper introduces ROOFS, a Python package that benchmarks multiple feature-selection (FS) methods on a user-provided dataset, reporting optimism-corrected predictive performance, stability, and true/false positive rates from a semi-synthetic outcome simulation. The authors apply ROOFS to the PIONeeR NSCLC immunotherapy dataset (n=435, p=374, reduced to p=214 after VIF filtering), comparing 23 FS methods combined with 11 classifiers. They report that a filter based on the union of Benjamini-Hochberg-adjusted p-values from t-test and logistic regression ('p.adjust') achieves the best optimism-corrected AUC (0.72), stability (S=0.39), and semi-synthetic TPR (0.93±0.06), outperforming LASSO and other embedded/wrapper methods. The paper argues that simple statistical filters can be more stable and comparably predictive than popular ML-based embedded methods in n≈p biomedical data.

Significance. The ROOFS package is a useful open-source contribution: it integrates a broad set of FS methods, automatic reporting, stability assessment, optimism correction, and semi-synthetic discovery evaluation, and the PIONeeR case study addresses a realistic heterogeneous n≈p clinical dataset. If the empirical claims survive scrutiny, the work supports a practical recommendation that simple statistical filters deserve a place in the standard toolkit. However, the central claim that p.adjust is the optimal method depends heavily on a semi-synthetic ground truth that is constructed from the same type of p-value screening as the recommended method; this circularity must be resolved before the empirical conclusions can be accepted.

major comments (3)
  1. [2.1.3, Table 2] The semi-synthetic 'true' predictor set S is defined as the 19 features with BH-adjusted p-values <0.01 from a multivariable Cox model fitted to the same PIONeeR data, and the simulated outcome is generated by a linear logistic function of these features. The method later recommended (§3.5) is itself a BH-based p-value filter (union of t-test and logistic regression at 0.05). Consequently, the TPR=0.93 reported in Table 2 is largely a self-consistency check: a p-value filter will, by construction, re-discover features selected by p-value thresholding on the same design matrix. The discussion in §2.2.5 acknowledges the difference in subset size between fixed- and variable-size methods but does not address this circularity. Please re-run the semi-synthetic benchmark with an independent construction of the ground truth (e.g., features selected at random, or from external biological criteria
  2. [3.5, Table 2] Even apart from the semi-synthetic issue, the real-outcome evidence for declaring p.adjust 'optimal' is thin: AUC 0.72 vs 0.70 for Fisher score and t-score, and stability 0.39 vs 0.36–0.37. No confidence intervals or pairwise significance tests are provided for these differences, so the superiority could be within bootstrap variability. Since §3.5 explicitly states that the choice was driven by predictive performance, stability, and discovery capability (TPR), and the TPR evidence is affected by the issue above, please supply uncertainty estimates (e.g., bootstrap CIs for AUC and stability, paired tests across classifiers) and adjust the strength of the conclusion accordingly.
  3. [2.2.5, Fig. 4] The TPR/FPR comparison is not adjusted for the number of features selected. p.adjust selects on average many more features than the fixed-size methods (up to 72 in bootstrap, vs k=19; see §3.5 and Figure 5C), which mechanically increases TPR and FPR. The paper notes this in §2.2.5 but still treats high TPR as evidence of discovery power. To support a 'best balance' claim, present a size-controlled analysis (e.g., TPR and FPR as functions of the selection threshold, or precision-recall curves) so that methods are compared at comparable signature sizes.
minor comments (4)
  1. [Table 2] The annotation 'Corresponds to better than LASSO' appears incomplete; the table does not clearly show which cells are being compared. Also, the 'Full data' row is formatted differently from the other rows and should be clarified.
  2. [3.4, Table 2] The text states that p.adjust FPR is significantly higher than random FS (0.1 vs 0.07), but Table 2 lists two random FS rows: fixed-size (FPR 0.09) and random-size (FPR 0.06). Please specify which random FS baseline is used in the comparison.
  3. [2.2.1 / Supplementary Table S1] The p.adjust filter is described as 'controlling for user-specified covariates; PD-L1 in the case of PIONeeR data', but the statistical model is not fully specified. State whether PD-L1 was included as a covariate in a univariate logistic regression for each feature, and whether this adjustment was applied consistently in every bootstrap fit.
  4. [Abstract / Section 1] The phrase 'low to zero additional programming' is unclear: users still need to prepare data and configure the pipeline. Consider rewording to describe the actual user effort required.

Circularity Check

1 steps flagged · score 6.0 of 10

Semi-synthetic ground truth is built from BH-adjusted p-values on the same data, and the winning p.adjust method is also a BH p-value filter; its TPR advantage is partly a benchmark artifact.

  1. fitted input called prediction [Section 2.1.3 (simulated outcome), Section 2.2.1 / Supplementary Table S1, Section 3.4 / Table 2]
    "For the benchmark on PIONeeR data, 19 true predictors were selected from the full dataset using a 0.01 threshold on Benjamini-Hochberg adjusted p-values from a multivariable Cox model for progression-free survival. p.adjust filter achieved the highest TPR (0.93 ± 0.06), rarely omitting true predictors. Union-based filter that combines features selected by t-test and logistic regression ... p-values from each test were adjusted using the Benjamini-Hochberg procedure with a 0.05 threshold."

    The 'true' feature set S is defined by BH-adjusted p-value screening on the same PIONeeR feature matrix, and the method crowned on TPR (p.adjust) is itself a BH-adjusted p-value screen (t-test/logistic). The synthetic outcome is then generated linearly from S, so features in S are, by construction, the ones any univariate p-value filter is best positioned to rank first. Although the ground-truth screen used a Cox model while p.adjust uses t-test/logistic, all are association p-values with the same BH correction, and on correlated data the selected sets heavily overlap. p.adjust's TPR=0.93±0.06 is therefore not an independent measurement of discovery ability but a partly tautological consequence of defining 'true' features through the same statistic family used by the winning method. The re

full rationale

The paper is a benchmark/tool paper. Its central derivation chain is: construct a semi-synthetic outcome from pre-selected 'true' features; run 23 FS methods; compare AUC, stability, TPR/FPR; declare p.adjust optimal. The real-outcome part (AUC and stability on the post-VIF PIONeeR data) is a self-contained comparison and does not reduce to the method's own outputs, giving p.adjust some independent support over LASSO. The circularity is concentrated in the semi-synthetic TPR: the ground-truth feature set was selected by BH-adjusted p-values on the same features, and the winning method is also a BH-adjusted p-value filter, so p.adjust's TPR of 0.93 is substantially built into the benchmark rather than independently demonstrated. The paper explicitly notes a related confound in Section 2.2.5 — fixed-size methods were capped at 19 features while variable-size p.adjust could select up to 72 — but it does not address the deeper issue that the ground-truth set itself was p-value-selected. Because AUC/stability still provide some independent content, the overall circularity is partial, not total: score 6.

Assumptions & free parameters 9 free parameters · 6 assumptions · 0 invented entities

The central benchmark rests on many user-set thresholds and simulation choices; the most consequential are VIF=5, BH thresholds 0.05/0.01, k=19 in the semi-synthetic arm, and the linear synthetic outcome. No new physical entities are postulated.

free parameters (9)
  • VIF threshold = 5
    Reduces 374 features to 214; all reported results are on the post-VIF dataset (Sec 2.1.2). Different thresholds would likely change stability and method rankings.
  • BH threshold for p.adjust filter = 0.05
    The winning method selects features whose BH-adjusted t-test or logistic-regression p-value is below 0.05 (Sec 2.1.3, Table S1).
  • BH threshold for semi-synthetic true predictors = 0.01
    The 19 'true' features were selected from the full data using BH-adjusted p-values < 0.01 from a Cox model (Sec 2.1.3).
  • Size of true predictor set S = 19
    Determined by the threshold above; TPR/FPR results depend on which and how many features are defined as true (Sec 2.1.3).
  • Synthetic outcome coefficients and noise = β estimated by unpenalized logistic regression; ε~N(0,1)
    Signal-to-noise of the semi-synthetic benchmark is set by these choices and determines how easily true features can be discovered (Sec 2.1.3).
  • Fixed selection size k=19 for fixed-size FS methods = 19
    In the semi-synthetic experiment, fixed-size methods were forced to select 19 features, capping their FPR and potentially raising TPR (Sec 2.2.5).
  • Bootstrap replicates and noise replicates = B=100; 40
    Stability and TPR/FPR estimates are Monte Carlo quantities; these values affect the precision of all reported estimates (Secs 2.2, 2.2.5).
  • Classifier hyperparameters = max_depth=2, min_samples_leaf=40, max_features=0.7, subsample=0.7 (Table S2)
    Downstream AUC rankings depend on these chosen hyperparameters; they were not tuned per FS method and constrain the models.
  • PD-L1 adjustment in logistic filter = PD-L1 covariate
    The winning logistic-regression p-values are adjusted for PD-L1; this covariate choice affects the selected signature (Table S1).
assumptions (6)
  • domain assumption BH procedure controls FDR under the correlation structure of the post-VIF data
    The paper itself cites Kanduri et al. showing that BH can be anti-conservative on intra-correlated data, and the post-VIF data still have residual correlation (Sec 3.4).
  • ad hoc to paper Semi-synthetic logistic-linear outcome is a meaningful proxy for the real outcome's feature-selection difficulty
    The simulation in Sec 2.1.3 uses a linear logistic model only, which favors linear filters; no evidence shows this replicates the real biological signal structure.
  • domain assumption Median/mode imputation and z-score normalization preserve true feature-outcome associations
    30.8% of values are missing; imputation is applied before FS in every bootstrap (Sec 2.2), and no sensitivity analysis is reported.
  • domain assumption VIF≤5 prefiltering retains all true predictive features
    If a true biomarker is removed because it is highly correlated with another feature, the benchmark can no longer assess it (Sec 2.1.2).
  • domain assumption .632+ optimism correction gives low-bias performance estimates for these pipelines
    The paper relies on ref [41] for this; its applicability to 23 FS methods × 11 classifiers on n=435 is assumed, not demonstrated.
  • standard math Nogueira stability formula is a valid basis for comparing FS methods
    Used as the stability metric in Sec 2.2.4; the lower bound depends on B and S=0 is interpreted as random selection.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ROOFS: RObust biOmarker Feature Selection." pith.science (2026). https://pith.science/paper/MLFL2YFX

@misc{pith2026260105151,
  author       = {Pith},
  title        = {Pith review of: ROOFS: RObust biOmarker Feature Selection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MLFL2YFX}},
  note         = {Machine review of arXiv:2601.05151}
}
read the original abstract

Feature selection (FS) is essential for biomarker discovery and clinical predictive modeling. Over the past decades, methodological literature on FS has become rich and mature, offering a wide spectrum of algorithmic approaches. However, much of this methodological progress has not fully translated into applied biomedical research. Moreover, challenges inherent in biomedical data, such as high-dimensional feature space, low sample size, multicollinearity, and missing values, make FS non-trivial. To help bridge this gap between methodological development and practical application, we propose ROOFS (RObust biOmarker Feature Selection), a Python package available at https://gitlab.inria.fr/compo/roofs, designed to help researchers in the choice of FS method adapted to their problem. ROOFS benchmarks multiple FS methods on the user's data and generates reports summarizing a comprehensive set of evaluation metrics, including downstream predictive performance estimated using optimism correction, stability, robustness of individual features, and true positive and false positive rates assessed on semi-synthetic data with a simulated outcome. We demonstrate the utility of ROOFS on data from the PIONeeR clinical trial, aimed at identifying predictors of resistance to anti-PD-(L)1 immunotherapy in lung cancer. Of the 34 FS methods gathered in ROOFS, we evaluated 23 in combination with 11 classifiers (253 models) and identified a filter based on the union of Benjamini-Hochberg false discovery rate-adjusted p-values from t-test and logistic regression as the optimal approach, outperforming other methods including widely used LASSO. We conclude that comprehensive benchmarking with ROOFS has the potential to improve the reproducibility of FS discoveries and increase the translational value of clinical models.

Figures

Figures reproduced from arXiv: 2601.05151 by the authors.

Figure 1
Figure 1. Overview of the ROOFS pipeline for comprehensive FS benchmarking. 2.2 Suggested framework The evaluation of a given FS method by ROOFS is based on a bootstrap experiment ( [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Instability of LASSO on the full PIONeeR dataset (p [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Improvement in FS stability following multicollinearity reduction with VIF pre-filtering compared to the [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Performance of representative FS methods from different algorithmic families (top: FS methods with fixed, [PITH_FULL_IMAGE:figures/full_fig_p010_4.png]
Figure 5
Figure 5. Figure 5: Comparison of performance between Shapicant (representative wrapper method), LASSO (baseline embedded [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Multimodality Stacking with Blockwise missing values and application to the PIONeeR biomarkers study for prediction of resistance to immunotherapy

    stat.AP 2026-05 unverdicted novelty 6.0 of 10

    MSB is a late-fusion stacking framework for multimodal survival prediction under blockwise missingness that improves C-index over baselines on the PIONeeR lung cancer immunotherapy dataset.

Reference graph

Works this paper leans on

59 extracted references · 17 canonical work pages · cited by 1 Pith paper

  1. [1]

    An introduction to variable and feature selection.Journal of Machine Learning Research, 3(Mar):1157–1182, 2003

    Isabelle Guyon and André Elisseeff. An introduction to variable and feature selection.Journal of Machine Learning Research, 3(Mar):1157–1182, 2003. ISSN ISSN 1533-7928

    2003

  2. [2]

    Trevino, Jiliang Tang, and Huan Liu

    Jundong Li, Kewei Cheng, Suhang Wang, Fred Morstatter, Robert P. Trevino, Jiliang Tang, and Huan Liu. Feature selection: A data perspective.ACM Computing Surveys, 50(6):94:1–94:45, 2017. ISSN 0360-0300. doi: 10.1145/3136625

    doi:10.1145/3136625 2017

  3. [3]

    Stability of feature selection algorithms

    Alexandros Kalousis, Julien Prados, and Melanie Hilario. Stability of feature selection algorithms. InProceedings of the Fifth IEEE International Conference on Data Mining, ICDM ’05, page 218–225, USA, 2005. IEEE Computer Society. ISBN 0769522785. doi: 10.1109/ICDM.2005.135. URL https://doi.org/10.1109/ ICDM.2005.135

    work page doi:10.1109/icdm.2005.135 2005

  4. [4]

    The influence of feature selection methods on accuracy, stability and interpretability of molecular signatures.PLOS ONE, 6(12):e28210, 2011

    Anne-Claire Haury, Pierre Gestraud, and Jean-Philippe Vert. The influence of feature selection methods on accuracy, stability and interpretability of molecular signatures.PLOS ONE, 6(12):e28210, 2011. ISSN 1932-6203. doi: 10.1371/journal.pone.0028210

    work page doi:10.1371/journal.pone.0028210 2011

  5. [5]

    Variable selection – a review and recommendations for the practicing statistician.Biometrical Journal, 60(3):431–449, 2018

    Georg Heinze, Christine Wallisch, and Daniela Dunkler. Variable selection – a review and recommendations for the practicing statistician.Biometrical Journal, 60(3):431–449, 2018. ISSN 1521-4036. doi: 10.1002/bimj.201700067

    work page doi:10.1002/bimj.201700067 2018

  6. [6]

    Benchmark for filter methods for feature selection in high-dimensional classification data.Computational Statistics & Data Analysis, 143:106839, 2020

    Andrea Bommert, Xudong Sun, Bernd Bischl, Jörg Rahnenführer, and Michel Lang. Benchmark for filter methods for feature selection in high-dimensional classification data.Computational Statistics & Data Analysis, 143:106839, 2020. ISSN 0167-9473. doi: https://doi.org/10.1016/j.csda.2019.106839. URL https://www. sciencedirect.com/science/article/pii/S0167947...

    arXiv 2020

  7. [7]

    Benchmarking feature projection methods in radiomics.Scientific Reports, 15(1):32368, sept

    Aydin Demircio˘glu. Benchmarking feature projection methods in radiomics.Scientific Reports, 15(1):32368, sept

  8. [8]

    Effect of subset size on the stability of feature selection for gene expression data

    Shaveta Tatwani and Ela Kumar. Effect of subset size on the stability of feature selection for gene expression data. InLecture Notes in Engineering and Computer Science: Proceedings of The World Congress on Engineering 2019, pages 541–546, London, U.K., July 2019

    2019

Show all 59 references
  1. [9]

    Evaluation of clinical prediction models (part 1): from development to external validation.BMJ, 384, 2024

    Gary S Collins, Paula Dhiman, Jie Ma, Michael M Schlussel, Lucinda Archer, Ben Van Calster, Frank E Harrell, Glen P Martin, Karel G M Moons, Maarten van Smeden, Matthew Sperrin, Garrett S Bullock, and Richard D Riley. Evaluation of clinical prediction models (part 1): from dev...

    2024 doi

  2. [10]

    Filter methods for feature selection in supervised machine learning applications - review and benchmark.ArXiv, abs/2111.12140, 2021

    Konstantin Hopf and Sascha Reifenrath. Filter methods for feature selection in supervised machine learning applications - review and benchmark.ArXiv, abs/2111.12140, 2021. URL https://api.semanticscholar. org/CorpusID:244527455

    2021 arXiv

  3. [11]

    Rintala, and Vittorio Fortino

    Luca Cattelani, Arindam Ghosh, Teemu J. Rintala, and Vittorio Fortino. A comprehensive evaluation framework for benchmarking multi-objective feature selection in omics-based biomarker discovery.IEEE/ACM Transactions on Computational Biology and Bioinformatics, 21(6):2432–2446,...

    2024

  4. [12]

    Justine Labory, Evariste Njomgue-Fotso, and Silvia Bottini. Benchmarking feature selection and feature extraction methods to improve the performances of machine-learning algorithms for patient classification using metabolomics biomedical data.Computational and Structural Biote...

    2024

  5. [13]

    Sugam Budhraja, Maryam Doborjeh, Balkaran Singh, Samuel Tan, Zohreh Doborjeh, Edmund Lai, Alexander Merkin, Jimmy Lee, Wilson Goh, and Nikola Kasabov. Filter and wrapper stacking ensemble (fwse): a robust approach for reliable biomarker discovery in high-dimensional omics data...

    2023 doi

  6. [14]

    Benchmark study of feature selection strategies for multi-omics data.BMC Bioinformatics, 23(1):412, October 2022

    Yingxia Li, Ulrich Mansmann, Shangming Du, and Roman Hornung. Benchmark study of feature selection strategies for multi-omics data.BMC Bioinformatics, 23(1):412, October 2022. ISSN 1471-2105. doi: 10.1186/ s12859-022-04962-x

    2022

  7. [15]

    Benchmark of filter methods for feature selection in high-dimensional gene expression survival data.Briefings in Bioinformatics, 23(1):bbab354, January 2022

    Andrea Bommert, Thomas Welchowski, Matthias Schmid, and Jörg Rahnenführer. Benchmark of filter methods for feature selection in high-dimensional gene expression survival data.Briefings in Bioinformatics, 23(1):bbab354, January 2022. ISSN 1477-4054. doi: 10.1093/bib/bbab354

    2022 doi

  8. [16]

    Comparison of five supervised feature selection algorithms leading to top features and gene signatures from multi-omics data in cancer.BMC Bioinformatics, 23 (3):153, April 2022

    Tapas Bhadra, Saurav Mallik, Neaj Hasan, and Zhongming Zhao. Comparison of five supervised feature selection algorithms leading to top features and gene signatures from multi-omics data in cancer.BMC Bioinformatics, 23 (3):153, April 2022. ISSN 1471-2105. doi: 10.1186/s12859-0...

    2022 doi

  9. [17]

    Ensemble feature selection in medical datasets: Combining filter, wrapper, and embedded feature selection results.Expert Systems, 37(5):e12553, October 2020

    Chih-Wen Chen, Yi-Hong Tsai, Fang-Rong Chang, and Wei-Chao Lin. Ensemble feature selection in medical datasets: Combining filter, wrapper, and embedded feature selection results.Expert Systems, 37(5):e12553, October 2020. ISSN 0266-4720. doi: 10.1111/exsy.12553

    2020 doi

  10. [18]

    An integrative multimodal machine learning signature of primary resistance to immunotherapy in advanced non-small cell lung cancer: biomarker analysis from the pioneer study

    Fabrice Barlesi, Florence Monville, Laurent Greillier, Natalie Ngoi, Joseph Ciccolini, Florence Sabatier, Stephane Garcia, Jean-Philippe Dales, Cyril Foa, Laurent Arnaud, Amélie Pouchin, Frédéric Vely, Sivan Bokobza, Anastasiia Bakhmach, Andrea Vaglio, Mélanie Karlsen, Paul Du...

    2026

  11. [19]

    Naiyer Rizvi, Foluso O Ademuyiwa, Z Alexander Cao, Helen X Chen, Robert L Ferris, Sarah B Goldberg, Matthew D Hellmann, Ranee Mehra, Ina Rhee, Jong Chul Park, Harriet Kluger, Hussein Tawbi, and Ryan J Sullivan. Society for immunotherapy of cancer (sitc) consensus definitions f...

  12. [20]

    Springer Publishing Company, Incorporated, 2017

    Gareth James, Daniela Witten, Trevor Hastie, and Robert Tibshirani.An Introduction to Statistical Learning: with Applications in R. Springer Publishing Company, Incorporated, 2017. ISBN 978-1-4614-7137-0

    2017

  13. [21]

    Pedregosa, G

    F. Pedregosa, G. Varoquaux, A. Gramfort, V . Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V . Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python.Journal of Machine Learnin...

    2011

  14. [22]

    Gaudillière, Francois-Xavier Ladant, Franck Verdonk, Ina A

    Julien Hédou, Ivana Mari ´c, Grégoire Bellan, Jakob Einhaus, Dyani K. Gaudillière, Francois-Xavier Ladant, Franck Verdonk, Ina A. Stelzer, Dorien Feyaerts, Amy S. Tsai, Edward A. Ganio, Maximilian Sabayev, Joshua Gillard, Jonas Amar, Amelie Cambriel, Tomiko T. Oskotsky, Alenni...

    2024

  15. [23]

    Regression shrinkage and selection via the lasso.Journal of the Royal Statistical Society

    Robert Tibshirani. Regression shrinkage and selection via the lasso.Journal of the Royal Statistical Society. Series B (Methodological), 58(1):267–288, 1996. ISSN 0035-9246

    1996

  16. [24]

    The adaptive lasso and its oracle properties.Journal of the American Statistical Association, 101(476): 1418–1429, December 2006

    Hui Zou. The adaptive lasso and its oracle properties.Journal of the American Statistical Association, 101(476): 1418–1429, December 2006. ISSN 0162-1459. doi: 10.1198/016214506000000735

    2006 doi

  17. [25]

    Exclusive lasso for multi-task feature selection

    Yang Zhou, Rong Jin, and Steven Chu-Hong Hoi. Exclusive lasso for multi-task feature selection. InProceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, page 988–995. JMLR Workshop and Conference Proceedings, March 2010. URLhttps://pr...

    2010

  18. [26]

    Francis R. Bach. Bolasso: model consistent lasso estimation through the bootstrap. InProceedings of the 25th International Conference on Machine Learning, ICML ’08, page 33–40, New York, NY , USA, 2008. Association for Computing Machinery. ISBN 978-1-60558-205-4. doi: 10.1145/...

    2008

  19. [27]

    Stability selection.Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(4):417–473, 2010

    Nicolai Meinshausen and Peter Bühlmann. Stability selection.Journal of the Royal Statistical Society: Series B (Statistical Methodology), 72(4):417–473, 2010. ISSN 1467-9868. doi: 10.1111/j.1467-9868.2010.00740.x

    2010

  20. [28]

    Rent—repeated elastic net technique for feature selection.IEEE Access, 9:152333–152346, 2021

    Anna Jenul, Stefan Schrunner, Kristian Hovde Liland, Ulf Geir Indahl, Cecilia Marie Futsæther, and Oliver Tomic. Rent—repeated elastic net technique for feature selection.IEEE Access, 9:152333–152346, 2021. doi: 10.1109/ACCESS.2021.3126429

    2021

  21. [29]

    Ultra high-dimensional nonlinear feature selection for big biological data.IEEE Transactions on Knowledge and Data Engineering, 30(7):1352–1365, July 2018

    Makoto Yamada, Jiliang Tang, Jose Lugo-Martinez, Ermin Hodzic, Raunak Shrestha, Avishek Saha, Hua Ouyang, Dawei Yin, Hiroshi Mamitsuka, Cenk Sahinalp, Predrag Radivojac, Filippo Menczer, and Yi Chang. Ultra high-dimensional nonlinear feature selection for big biological data.I...

    2018

  22. [30]

    Conditional infomax learning: An integrated framework for feature extraction and fusion

    Dahua Lin and Xiaoou Tang. Conditional infomax learning: An integrated framework for feature extraction and fusion. In Aleš Leonardis, Horst Bischof, and Axel Pinz, editors,Computer Vision – ECCV 2006, pages 68–82, Berlin, Heidelberg, 2006. Springer Berlin Heidelberg. ISBN 978...

    2006

  23. [31]

    Fast binary feature selection with conditional mutual information.J

    François Fleuret. Fast binary feature selection with conditional mutual information.J. Mach. Learn. Res., 5: 1531–1555, December 2004. ISSN 1532-4435. 15 APREPRINT- FEBRUARY16, 2026

    2004

  24. [32]

    Meyer and Gianluca Bontempi

    Patrick E. Meyer and Gianluca Bontempi. On the use of variable complementarity for feature selection in cancer classification. In Franz Rothlauf, Jürgen Branke, Stefano Cagnoni, Ernesto Costa, Carlos Cotta, Rolf Drechsler, Evelyne Lutton, Penousal Machado, Jason H. Moore, Juan...

    2006

  25. [33]

    Data visualization and feature selection: New algorithms for nongaussian data

    Howard Yang and John Moody. Data visualization and feature selection: New algorithms for nongaussian data. In S. Solla, T. Leen, and K. Müller, editors,Advances in Neural Information Processing Systems, vol- ume 12. MIT Press, 1999. URL https://proceedings.neurips.cc/paper_fil...

    1999

  26. [34]

    Theoretical and empirical analysis of relieff and rrelieff.Machine Learning, 53(1):23–69, October 2003

    Marko Robnik-Šikonja and Igor Kononenko. Theoretical and empirical analysis of relieff and rrelieff.Machine Learning, 53(1):23–69, October 2003. ISSN 1573-0565. doi: 10.1023/A:1025667309714

    2003 doi

  27. [35]

    Controlling the false discovery rate: A practical and powerful approach to multiple testing.Journal of the Royal Statistical Society

    Yoav Benjamini and Yosef Hochberg. Controlling the false discovery rate: A practical and powerful approach to multiple testing.Journal of the Royal Statistical Society. Series B (Methodological), 57(1):289–300, 1995. ISSN 0035-9246

    1995

  28. [36]

    Shapicant: feature selection package based on shap and target permutation

    Manuel Calzolari. Shapicant: feature selection package based on shap and target permutation. https://github. com/manuel-calzolari/shapicant, 2020. Version 0.4.0

    2020

  29. [37]

    F. E. Harrell, K. L. Lee, and D. B. Mark. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.Statistics in Medicine, 15(4):361–387, February

  30. [38]

    Estimating the error rate of a prediction rule: Improvement on cross-validation.Journal of the American Statistical Association, 78(382):316–331, 1983

    Bradley Efron. Estimating the error rate of a prediction rule: Improvement on cross-validation.Journal of the American Statistical Association, 78(382):316–331, 1983. ISSN 0162-1459. doi: 10.2307/2288636

    1983 doi

  31. [39]

    Improvements on cross-validation: The .632+ bootstrap method.Journal of the American Statistical Association, 92(438):548–560, 1997

    Bradley Efron and Robert Tibshirani. Improvements on cross-validation: The .632+ bootstrap method.Journal of the American Statistical Association, 92(438):548–560, 1997. ISSN 0162-1459. doi: 10.2307/2965703

    1997 doi

  32. [40]

    Chernick and Robert A

    Michael R. Chernick and Robert A. LaBudde.An Introduction to Bootstrap Methods with Applications to R. Wiley Publishing, 1st edition, October 2011. ISBN 978-0-470-46704-6

    2011

  33. [41]

    Katsuhiro Iba, Tomohiro Shinozaki, Kazushi Maruo, and Hisashi Noma. Re-evaluation of the comparative effectiveness of bootstrap-based optimism correction methods in the development of multivariable clinical prediction models.BMC Medical Research Methodology, 21(1):9, January 2...

    2021 doi

  34. [42]

    On the stability of feature selection algorithms.Journal of Machine Learning Research, 18(174):1–54, 2018

    Sarah Nogueira, Konstantinos Sechidis, and Gavin Brown. On the stability of feature selection algorithms.Journal of Machine Learning Research, 18(174):1–54, 2018. ISSN 1533-7928

    2018

  35. [43]

    On model selection consistency of lasso.Journal of Machine Learning Research, 7(90): 2541–2563, 2006

    Peng Zhao and Bin Yu. On model selection consistency of lasso.Journal of Machine Learning Research, 7(90): 2541–2563, 2006. ISSN 1533-7928

    2006

  36. [44]

    Beware of counter-intuitive levels of false discoveries in datasets with strong intra-correlations.Genome Biology, 26(1):249, August 2025

    Chakravarthi Kanduri, Maria Mamica, Emilie Willoch Olstad, Manuela Zucknick, Jingyi Jessica Li, and Geir Kjetil Sandve. Beware of counter-intuitive levels of false discoveries in datasets with strong intra-correlations.Genome Biology, 26(1):249, August 2025. ISSN 1474-760X. do...

    2025 doi

  37. [45]

    Riley and Gary S

    Richard D. Riley and Gary S. Collins. Stability of clinical prediction models developed using statistical or machine learning methods.Biometrical Journal, 65(8):2200302, 2023. ISSN 1521-4036. doi: 10.1002/bimj.202200302

    2023 doi

  38. [46]

    IntechOpen, February 2010

    Petr Somol, Jana Novovicova, and Pavel Pudil.Efficient Feature Subset Selection and Subset Size Optimization. IntechOpen, February 2010. ISBN 978-953-7619-90-9. doi: 10.5772/9356. URL https://www.intechopen. com/chapters/10666

    2010 doi

  39. [47]

    Drotár, J

    P. Drotár, J. Gazda, and Z. Smékal. An experimental comparison of feature selection methods on two-class biomedical datasets.Computers in Biology and Medicine, 66:1–10, November 2015. ISSN 0010-4825. doi: 10.1016/j.compbiomed.2015.08.010

    2015 doi

  40. [48]

    The schism study: Circulating cell-free dna size profiles as predictors of progression in advanced carcinoma treated with immune-checkpoint inhibitors

    Linh Nguyen Phuong, Frederic Fina, Laurent Greillier, Pascale Tomasini, Jean-Laurent Deville, Romain Zakrasjek, Lucie Della-Negra, Audrey Boutonnet, Frédéric Ginot, Jean-Charles Garcia, Sébastien Benzekry, and Sébastien Salas. The schism study: Circulating cell-free dna size p...

    2026

  41. [49]

    Nicolas Captier, Marvin Lerousseau, Fanny Orlhac, Narinée Hovhannisyan-Baghdasarian, Marie Luporsi, Erwin Woff, Sarah Lagha, Paulette Salamoun Feghali, Christine Lonjou, Clément Beaulaton, Andrei Zinovyev, Hélène Salmon, Thomas Walter, Irène Buvat, Nicolas Girard, and Emmanuel...

    2026

  42. [50]

    Schism: Mechanistic modeling of cfdna fragmen- tome dynamics predicts progression to immunotherapy, December 2025

    Linh Nguyen Phuong, Audrey Boutonnet, Laurent Greillier, Pascale Tomasini, Jean-Laurent Deville, Sebastien Salas, Frederic Ginot, Frédéric Fina, and Sébastien Benzekry. Schism: Mechanistic modeling of cfdna fragmen- tome dynamics predicts progression to immunotherapy, December...

    2025

  43. [51]

    Block hsic lasso: model-free biomarker detection for ultra-high dimensional data.Bioinformatics (Oxford, England), 35(14): i427–i435, july 2019

    Héctor Climente-González, Chloé-Agathe Azencott, Samuel Kaski, and Makoto Yamada. Block hsic lasso: model-free biomarker detection for ultra-high dimensional data.Bioinformatics (Oxford, England), 35(14): i427–i435, july 2019. ISSN 1367-4811 1367-4803. doi: 10.1093/bioinformat...

    2019 doi

  44. [52]

    https://scikit-learn.org/stable/ auto_examples/inspection/plot_permutation_importance_multicollinear.html

    Permutation Importance with Multicollinear or Correlated Features. https://scikit-learn.org/stable/ auto_examples/inspection/plot_permutation_importance_multicollinear.html

  45. [53]

    Gene selection for cancer classification using support vector machines.Machine Learning, 46:389–422, January 2002

    Isabelle Guyon, Jason Weston, Stephen Barnhill, and Vladimir Vapnik. Gene selection for cancer classification using support vector machines.Machine Learning, 46:389–422, January 2002. doi: 10.1023/A:1012487302797

    2002 doi

  46. [54]

    Joint analysis of two microarray gene-expression data sets to select lung adenocarcinoma marker genes.BMC Bioinformatics, 5(1):81, june 2004

    Hongying Jiang, Youping Deng, Huann-Sheng Chen, Lin Tao, Qiuying Sha, Jun Chen, Chung-Jui Tsai, and Shuanglin Zhang. Joint analysis of two microarray gene-expression data sets to select lung adenocarcinoma marker genes.BMC Bioinformatics, 5(1):81, june 2004. ISSN 1471-2105. do...

    2004 doi

  47. [55]

    An iterative model-free feature screening procedure: Forward recursive selection

    Siwei Xia and Yuehan Yang. An iterative model-free feature screening procedure: Forward recursive selection. Knowledge-Based Systems, 246:108745, june 2022. ISSN 0950-7051. doi: 10.1016/j.knosys.2022.108745. 17 APREPRINT- FEBRUARY16, 2026 5 Supplementary Supplementary table S1...

    2022

  48. [370]

    URL https://www.sciencedirect.com/science/ article/pii/S2001037024000692

    doi: https://doi.org/10.1016/j.csbj.2024.03.016. URL https://www.sciencedirect.com/science/ article/pii/S2001037024000692

    2024 doi

  49. [1996]

    doi: 10.1002/(SICI)1097-0258(19960229)15:4%3C361::AID-SIM168%3E3.0.CO;2-4

    ISSN 0277-6715. doi: 10.1002/(SICI)1097-0258(19960229)15:4%3C361::AID-SIM168%3E3.0.CO;2-4

    doi

  50. [2023]

    doi: 10.1136/jitc-2022-005920

    ISSN 2051-1426. doi: 10.1136/jitc-2022-005920

    doi

  51. [2025]

    doi: 10.1038/s41598-025-16070-w

    ISSN 2045-2322. doi: 10.1038/s41598-025-16070-w

    doi

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.