Best Subset Selection: Optimal Pursuit for Feature Selection and Elimination

Yanhao Zhang; Yong Xia; Zhihan Zhu

REVIEW 3 major objections 4 minor 40 references

Best Subset Selection: Optimal Pursuit for Feature Selection and Elimination

T0 review · 3 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper proposes two objective-based criteria—one for adding features, one for removing them—that are provably optimal for the current support-set update, and shows that substituting them for classical criteria in greedy algorithms…

desk verdict Solid greedy-selection upgrade with credible experiments, but the CoSaOP convergence claim rides on an assumption that smuggles in the true support. read the letter →

arxiv 2501.16815 v3 pith:RBWSZ6SF submitted 2025-01-28 math.OC

Zhihan Zhu , Yanhao Zhang , Yong Xia This is my paper · ORCID

classification math.OC MSC 90C2790C5962J05

keywords bestsubsetselectionfeatureeliminationgreedyalgorithmscompressedsensingsparseregressionoptimalpursuitmeta-substitution

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is about the two choice points inside every greedy best-subset-selection algorithm: which feature to add and which feature to drop. The paper's central claim is that the classical rules—maximizing correlation with the residual for entry, minimizing a Wald/T-style statistic for exit—solve only a partial version of the update problem, because they ignore the coefficient refit that follows the support change. Solving the full one-entry and one-exit least-squares subproblems produces two new criteria (labeled (8) and (10)) that account for interactions among features, and Theorems 4.1 and 4.2 show that the chosen index truly minimizes the least-squares objective over all possible entries or exits from the current support. Using these criteria in place of the classical ones yields enhanced versions of OMP, CoSaMP, and (A)BESS—named OP, CoSaOP, and OP-(A)BESS—that the paper argues preserve the original convergence guarantees while delivering large gains in signal-recovery success, NMSE, R-squared, and runtime. A sympathetic reader would take the central contribution to be a general 'swap in better criteria' principle for subset selection, not just three improved algorithms.

What carries the argument

The load-bearing object is the support-restricted least-squares value $f(S)=\min_{\operatorname{supp}(\beta)=S}\|y-X\beta\|_2^2$, whose one-step behavior the new criteria optimize exactly. The mechanical engine is a pair of matrix-inverse update lemmas: the forward inverse (Lemma 3.2) writes $(X_S^{\top}X_S)^{-1}$ for $S=S_{k-1}\cup\{j\}$ as a rank-one update of the already-available inverse $C_{k-1}$, converting the exact selection subproblem into the explicit criterion (8); the backward inverse (Lemma 3.10) performs the matching downdate for $S=S_{k-1}\setminus\{j\}$, yielding criterion (10). These lemmas are what keep the new criteria at the same computational order as the classical ones: the required inverse or Cholesky factor was already computed when the coefficients on $S_{k-1}$ were fit, so the criteria add no new factorization cost.

What would settle it

Take a small design matrix and a sparse vector, enumerate $f(S\cup\{j\})$ for every $j\notin S$ and $f(S\setminus\{j\})$ for every $j\in S$ by fitting least squares on each candidate support, and check that the indices selected by criteria (8) and (10) attain the minima; the first violation would refute Theorems 4.1 and 4.2. For the convergence-preservation claim, construct a design matrix with restricted-isometry constant 0.1 where the true support intersects the disagreement set in the CoSaOP proof and run CoSaOP; a residual sequence that fails the claimed linear bound would refute Theorem 4.9 as stated.

Watch

Extended reading notes

Core claim

Stated on the paper's own terms, the discovery is that the classical selection and elimination rules of greedy subset selection are only first steps of block coordinate descent on the support-restricted least-squares objective $f(S)=\min_{\beta:\,\operatorname{supp}(\beta)=S}\|y-X\beta\|_2^2$. The classical entry rule (3) maximizes the immediate drop in $\|y-X\beta\|^2$ from adding $j$ with all other coefficients frozen, and the exit rule (4) minimizes the immediate rise from deleting $j$ with other coefficients frozen; the subsequent coefficient refit is invisible to both rules. The paper formulates the exact subproblems (P1) and (Q1), in which coefficients are refit after the support change, solves them in closed form, and obtains criterion (8) for entry and criterion (10) for exit. Theorem 4.1 then states that the index $j^*$ from (8) satisfies $f(S\cup\{j^*\})\le f(S\cup\{j\})$ for every candidate $j$ outside $S$, and Theorem 4.2 states the symmetric statement for (10) and deletion. Replacing criteria throughout existing algorithms yields OP, CoSaOP, and OP-(A)BESS; Theorem 4.9 gives CoSaOP a CoSaMP-style linear convergence rate under RIP, and Theorems 4.10 and 4.11 quantify advantages over the classical rules when features are correlated.

Load-bearing premise

The load-bearing premise is an extra disjointness condition in the CoSaOP proof—the true support must avoid every index where the elimination rule's output and the best K-sparse approximation to the current fit disagree—and the paper does not show this follows from its stated restricted-isometry bound.

Editorial extensions

If this is right

Theorems 4.1 and 4.2 make the one-step choices optimal: at a given support $S$, the selected entry or exit index produces the smallest possible support-restricted least-squares objective among all candidates.
Meta-substitution turns the paper's criteria into a general recipe: any greedy subset-selection algorithm that uses correlation-based entry or T-statistic exit can be upgraded, and the paper demonstrates this for OMP, CoSaMP, and (A)BESS.
Because both new criteria reuse the inverse already computed for the current support, OP, CoSaOP, and OP-(A)BESS stay in the same computational-complexity class as their classical counterparts.
Under restricted-isometry assumptions, CoSaOP inherits a CoSaMP-style linear convergence bound, with the constants the paper computes in Theorem 4.9.
In the correlated-feature regime where RIP fails, criterion (8) has a lower bound that grows as the correlation between two true features approaches one, while the classical criterion's upper bound shrinks to zero; criterion (10) can identify pseudo-correlated spurious features for removal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Inference: the entry criterion (8) is effectively correlation with the residual after the current support's contribution is projected out, so the same logic should transfer to matching-pursuit variants beyond the three tested families, such as LARS-style or kernel-based greedy fits; this is a direct extension the paper does not run.
Inference: the elimination criterion (10) is a support-agnostic pruning rule, so it could be used as a generic post-processing or inner-loop step in any local-search subset-selection solver, not only in greedy pipelines; the authors' 'meta-substitution' language suggests this, but the experiments only cover three families.
Inference: the Optimal Gradient Pursuit appendix hints that the exact-subproblem principle extends to non-quadratic objectives; a testable formulation would define entry and exit gains relative to gradient updates with exact line search and compare against current greedy methods on logistic or quantile losses.
Inference: the claimed same-order cost depends on the current-support inverse being available; in ultra-high-dimensional settings where even forming the projection in criterion (8) is prohibitive, the paper's own Optimal Gradient Pursuit version is the practical route, but its one-step optimality is not proved to the same standard as Theorem 4.1.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Bottom line: this is a useful paper, but the headline theory claim is not established. The optimization derivation of the selection and elimination criteria is correct, and the experiments are extensive and credible. The proof that CoSaOP preserves CoSaMP's linear convergence has a real gap that the authors do not flag in the main text.

What's genuinely new: the unified derivation of criteria (8) and (10) as exact solutions of the entry/exit subproblems, and the meta-substitution framework that upgrades OMP, CoSaMP, and ABESS. The reader is right that (8) is essentially the classical F-to-enter statistic and (10) the backward elimination statistic, and Remark 3.8 concedes near-overlap with SMP. Theorems 4.1 and 4.2 are correct but definitional; they restate what the criteria were built to do. That is a minor issue.

The stress-test note holds up. Theorem 4.9's chain relies on Lemma 4.7, whose proof uses Assumption 4 in Appendix I: supp(β*) ⊆ (S_k △ supp(ã_K))^c. This is not derived from RIP δ≤0.1, and because all three sets have cardinality K, it effectively forces S_k = supp(ã_K) and, when any true feature lies in U, S_k = supp(β*). That is, the assumption says the new elimination rule has already found the true support. Without it, a true feature in U can be pruned, and the bound ∥β*−ã_{S_k}∥ ≤ (2+δ)∥β*−a∥ fails. So the linear convergence rate for CoSaOP is not proven, and the abstract's 'preserve the theoretical properties' is unsupported for that algorithm. The paper needs to either prove Assumption 4 from RIP or state it as an extra condition and adjust the claim.

Second concern: 'without increasing computational cost' is overstated. Appendix O shows the new criteria take slightly more time; same order of magnitude, not zero overhead. That is a wording fix, not a deep flaw.

Third: no comparison with SMP. Given Remark 3.8, that is a notable omission.

Who this is for: people working on greedy best-subset selection, compressed sensing, and sparse regression. The algorithm family and experiments are valuable; OP and OP-(A)BESS look genuinely stronger on correlated problems, and the code is public. But the theory section needs work. I'd send it to review, with a request to fix the CoSaOP argument or clearly label Assumption 4 as an additional hypothesis, tone down the cost claim, and add the SMP comparison. I would not build on the CoSaOP theory as it stands.

Referee Report

3 major / 4 minor

Summary. The manuscript proposes two new criteria for best-subset selection: a selection criterion (8) and an elimination criterion (10), obtained by solving one-entry and one-exit least-squares subproblems exactly via forward and backward matrix-inversion formulas. The paper proves that these criteria give per-step optimal decisions (Theorems 4.1 and 4.2), claims that substituting them into OMP, CoSaMP, and (A)BESS preserves the original algorithms' theoretical guarantees while achieving substantial empirical gains, and reports experiments on compressed sensing, sparse regression, column subset selection, and line-spectrum estimation.

Significance. The algebraic derivations of (8) and (10) are clean, the per-step optimality theorems are correct, and the correlated-feature counterexample in Theorem 4.11 provides a concrete demonstration of a real failure mode of Wald-T-based pruning. The experimental evaluation is extensive, with multiple datasets, metrics, and runtime comparisons, and the authors provide code and detailed appendices. The contribution would be substantial if the guarantee-preservation claim were established. However, as detailed below, the preservation claim currently rests on an extra assumption about the new elimination rule, so the main theoretical selling point is not yet proven.

major comments (3)

[§4, Lemma 4.7 / Appendix I, Assumption 4] The convergence proof for CoSaOP is not established because Lemma 4.7 relies on Assumption 4 in Appendix I: supp(β*)⊆(S_k △ supp(ã_K))^c. This assumption is not derived from the RIP condition δ≤0.1 used in Lemma 4.4, nor from any earlier result. In the sparse-recovery setting S_k, supp(ã_K), and supp(β*) all have cardinality K, so in the generic case where a true index lies in S_k, the assumption forces S_k = supp(ã_K) and S_k = supp(β*); that is, it assumes that the new elimination criterion (10) has already recovered the true support and that pruning does not remove any true index. If a true feature with a large coefficient were dropped, then ∥β*−ã_{S_k}∥ could be proportional to that large coefficient even when a≈β*, so the bound ∥β*−ã_{S_k}∥ ≤ (2+δ)∥β*−a∥ of Lemma 4.7 can fail by an arbitrarily large factor. Consequently, Theorem 4.9's linear convergence rate for CoSaOP is not proven, and the Abstract's claim that enhanced algorithms 'preserve the theoretical properties of the original algorithms' is unsupported for CoSaOP, which is the paper's worked example of preservation.
[Abstract / Conclusion / §5.1, Appendix O] The claim that the enhanced algorithms achieve gains 'without increasing computational cost' overstates what is shown. Theorem 4.3 establishes only that the computational complexity is of the same order of magnitude, and Appendix O explicitly reports that the new criteria result in 'slightly higher computational time'. The abstract and conclusion should be reworded to claim 'same asymptotic order of complexity' rather than 'no increase in computational cost', so that the main text, the abstract, and the runtime data in Figure 5 are mutually consistent.
[§4, Theorem 4.11(2) / Appendix M] The proof of Theorem 4.11(2) ends with the sentence that criterion (10) 'will always identify Xp by Theorem 4.2'. This is stronger than what is proved. Theorem 4.2 states only that criterion (10) minimizes f(S\{j}) over j in the current S; it does not by itself guarantee that the minimizer is the pseudo-correlated feature in every configuration satisfying the stated correlation condition. The specific 3×3 counterexample is sufficient to demonstrate the 'could' claim of the theorem, but the 'always' phrasing should be removed or supported by a separate argument.

minor comments (4)

[§4, Theorems 4.1 and 4.2] Theorems 4.1 and 4.2 are immediate restatements of the derivations of (8) and (10); they should be explicitly described as per-step optimality statements rather than standalone convergence or recovery guarantees, to avoid overstating their role.
[Appendix F / Appendix G / Appendix H / Appendix I] The appendix headings refer to 'Theorem 4.4', 'Theorem 4.5', etc., while the main text labels these results as Lemmas 4.4–4.8; the numbering should be harmonized.
[Appendix O, Figure 5] The caption for Figure 5 should state explicitly whether the reported times are total wall-clock time over 500 runs or per-run averages, and the discussion of CoSaMP's larger runtime should be moved closer to the figure for clarity.
[§5.1.1, Appendix P] There is a typo 'signiaficant' in Section 5.1.1 and 'pseudo-correlated' is misspelled in Theorem 4.11(2); these should be corrected in the revision.

Circularity Check

1 steps flagged · score 2.0 of 10

No substantive circularity: The optimality theorems 4.1 and 4.2 restate the construction of criteria (8) and (10), but the empirical meta-gains and the convergence analysis are independent of any fitted or self-cited input.

self definitional [Section 4, Theorems 4.1 and 4.2; criteria constructed in Section 3.2, Eqs. (8) and (10)]
"Theorem 4.1. For index j ∗ selected by criterion(8), f(S∪ {j∗})≤f(S∪ {j}),∀j∈S c. Theorem 4.2. For index j ∗ selected by criterion(10), f(S\{j∗})≤f(S\{j}),∀j∈S."

Criterion (8) is derived by solving the exact one-step optimization (P1)/(P2): Lemma 3.1 rewrites (P2) as maximizing y^T X_S(X_S^T X_S)^{-1} X_S^T y over S=S_{k-1}∪{j}, i.e. minimizing f(S∪{j}); Theorem 3.3 identifies (8) as the argmax. Criterion (10) is likewise the solution of (Q1)/(Q2) in Theorem 3.11. Theorems 4.1 and 4.2 thus restate the defining optimization used to construct the criteria. The algebraic closed forms are genuine derivations, but the optimality assertions add no independent evidence beyond the construction; this is a minor definitional identity, not a fitted-input prediction.

full rationale

The paper's central empirical claim—that substituting criteria (8) and (10) into OMP, CoSaMP, and (A)BESS yields meta-gains across compressed sensing, sparse regression, column subset selection, and complex-signal tasks—is supported by external experiments against standard baselines and does not reduce to the criteria's definitions. No parameters are fitted and later renamed as predictions, and no self-citation chain is load-bearing: the ABESS and CoSaMP citations are external prior work, and the similarity to SMP (Tohidi et al. 2025) is disclosed in Remark 3.8. The only construction-relative step is the optimality claim in Theorems 4.1 and 4.2, which restates the exact subproblems used to define the criteria; because the paper explicitly frames these as solved subproblems, it is a definitional restatement rather than a hidden circularity. The main weakness is not circularity but proof completeness: Theorem 4.9's linear rate for CoSaOP relies on Assumption 4 in Appendix I, supp(β*)⊆(S_k △ supp(ã_K))^c, which is not shown to follow from the stated RIP condition δ≤0.1. That is a correctness gap in the preservation-of-guarantees claim, not a circular reduction, since the assumption is not equivalent to the theorem's inputs. Overall circularity is low.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No free parameters are fitted. The criteria are derived from the least-squares objective. The main extra load is Assumption 4 for the CoSaOP convergence proof. The paper introduces no new entities.

assumptions (4)

domain assumption Design matrix X satisfies the Restricted Isometry Property with δ_K ≤ δ_{K+1} ≤ δ_{2K} ≤ δ_{4K} ≤ 0.1, and columns are normalized to unit norm.
Invoked in the proofs of Lemmas 4.4, 4.6, 4.8 and Theorem 4.9 (Appendices F-H, J), inherited from CoSaMP theory.
ad hoc to paper Assumption 4 in Appendix I: supp(β*)⊆(S_k △ supp(ã_K))^c, where S_k is the support chosen by criterion (10) and ã_K is the best K-sparse approximation of the least-squares estimate on the merged support.
Needed to bound ||ã_K - ã_{S_k}||_2 in the proof of Lemma 4.7; not derived from the RIP assumptions and not stated in the main text.
standard math For every subset S considered, X_S has full column rank so (X_S^T X_S) is invertible.
Implicit in forming least-squares solutions and inverse update formulas throughout Section 3.
domain assumption In the synthetic compressed sensing experiments, the true support is unique and recoverable; experiments measure exact support recovery.
Section 5.1.1 defines successful recovery as S_hat = S*, which presumes a unique ground-truth support.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Best Subset Selection: Optimal Pursuit for Feature Selection and Elimination." pith.science (2026). https://pith.science/paper/RBWSZ6SF

@misc{pith2026250116815,
  author       = {Pith},
  title        = {Pith review of: Best Subset Selection: Optimal Pursuit for Feature Selection and Elimination},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RBWSZ6SF}},
  note         = {Machine review of arXiv:2501.16815}
}

read the original abstract

This paper introduces two novel criteria: one for feature selection and another for feature elimination in the context of best subset selection, which is a benchmark problem in statistics and machine learning. From the perspective of optimization, we revisit the classical selection and elimination criteria in traditional best subset selection algorithms, revealing that these classical criteria capture only partial variations of the objective function after the entry or exit of features. By formulating and solving optimization subproblems for feature entry and exit exactly, new selection and elimination criteria are proposed, proved as the optimal decisions for the current entry-and-exit process compared to classical criteria. Replacing the classical selection and elimination criteria with the proposed ones generates a series of enhanced best subset selection algorithms. These generated algorithms not only preserve the theoretical properties of the original algorithms but also achieve significant meta-gains without increasing computational cost across various scenarios and evaluation metrics on multiple tasks such as compressed sensing and sparse regression.

Figures

Figures reproduced from arXiv: 2501.16815 by the authors.

**Figure 1.** A (B) illustrates subset selection (elimination), with A1 (B1) comparing objectives and optimization problems of the old and new criteria, and A2 (B2) comparing function decreases (increases). argmin β ∥y − Xβ∥ 2 2 (P0) s.t. ∥β − β k−1 ∥0 ≤ 1, supp(β k−1 ) ⊂ supp (β). argmin β ∥y − Xβ∥ 2 2 (Q0) s.t. ∥β − β k−1 ∥0 ≤ 1, supp(β) ⊂ supp(β k−1 ). As the model evolves to (P0) and (Q0), the limitations of classic criteria … view at source ↗

**Figure 2.** Left: The original algorithmic workflow based on heuristic subset selection criteria. Right: By replacing the classic criteria with optimal objective-based criteria, the original heuristic algorithms can be updated to yield new algorithms accordingly. We classify subset selection algorithms into three categories based on their combination strategies for feature selection and elimination, providing one representati… view at source ↗

**Figure 3.** Meta-gain comparison of three kinds of subset selection algorithms. Row one: different sampling rates (SNR = 15). Row two: Varying SNRs (measurement rate = 0.25) [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (10 more)

**Figure 4.** Figure 4: Rows 1–3 present the meta-gains in feature representation capability (R 2 , closer to 1 is better) for the Boston Housing, California Housing, Superconductivity datasets, House 16H, Prostate.v8.egenes, and Spectra datasets, respectively, across three algorithms as the …

**Figure 5.** Figure 5: Running time for each algorithm over 500 independent runs. 2 In [PITH_FULL_IMAGE:figures/full_fig_p021_5.png]

**Figure 6.** Figure 6: Phase transition with correlated features. Q. Reasons Why CoSaMP Fails in Sparse Regression The main algorithmic flow of CoSaMP is as follows: it iteratively (1) selects 2K features, (2) solves a least squares problem on a large subset, and (3) prunes to K coefficients…

**Figure 7.** Figure 7: Reasons why CoSaMP fails in sparse regression. As shown in Figure 7a, on a regression dataset (Boston Housing) with highly correlated features, the pruned support set’s direct coefficients (column 2) differ significantly from those after least squares estimation (colum…

**Figure 8.** Figure 8: Residual evolution of CoSaMP with different feature correlation cases. In contrast, when features are weakly correlated (as shown in Figure 7b), the coefficients and residuals after pruning the large support set (column 2) and performing least squares (column 3) are ne…

**Figure 9.** Figure 9: Rows 1–3 present the meta-gains in cross-validation performance in prediction (errorpred, smaller is better) for the Boston Housing, California Housing, Superconductivity datasets, House 16H, Prostate.v8.egenes, and Spectra datasets, respectively, across three algorith…

**Figure 10.** Figure 10: The idea of Optimal Gradient Pursuit. Remark S.1. The criterion (28) is the OGP version of the OP feature selection criterion (8). Similarly, by applying techniques analogous to those in the Appendix T, we can derive the OGP version of the feature elimination criterio…

**Figure 11.** Figure 11: Residual convergence of Optimal Gradient Pursuit and Gradient Pursuit. Experiment on Computation Time: We further compared the runtime of GP and OGP, as shown in [PITH_FULL_IMAGE:figures/full_fig_p026_11.png]

**Figure 12.** Figure 12: Running time for each algorithm over 500 independent runs. V. Optimal Pursuit for Column Subset Selection Optimal Pursuit Criteria for Column Subset Selection: We can follow the procedure in Section 3 to construct the feature selection and elimination subproblems for …

**Figure 13.** Figure 13: Comparison of Optimal Pursuit enhanced algorithms and classic algorithms on complex signal processing. From the experimental results, it is evident that the enhanced algorithm achieves significantly lower frequency estimation error probability and higher correlation i…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

40 extracted references · 32 canonical work pages

[1]

A determinantal point process for column subset selection

Belhadji, A., Bardenet, R., and Chainais, P. A determinantal point process for column subset selection. Journal of Machine Learning Research, 21 0 (197): 0 1--62, 2020

work page 2020
[2]

and Davies, M

Blumensath, T. and Davies, M. E. Gradient pursuits. IEEE Transactions on Signal Processing, 56 0 (6): 0 2370--2382, 2008

work page 2008
[3]

and Davies, M

Blumensath, T. and Davies, M. E. Iterative hard thresholding for compressed sensing. Applied and Computational Harmonic Analysis, 27 0 (3): 0 265--274, 2009

work page 2009
[4]

S., Donoho, D

Chen, S. S., Donoho, D. L., and Saunders, M. A. Atomic decomposition by basis pursuit. SIAM Review, 43 0 (1): 0 129--159, 2001

work page 2001
[5]

and Kempe, D

Das, A. and Kempe, D. Submodular meets spectral: greedy algorithms for subset selection, sparse approximation and dictionary selection. In Proceedings of the 28th International Conference on International Conference on Machine Learning, pp.\ 1057–1064, 2011

work page 2011
[6]

and Kempe, D

Das, A. and Kempe, D. Approximate submodularity and its applications: Subset selection, sparse approximation and dictionary selection. Journal of Machine Learning Research, 19 0 (3): 0 1--34, 2018

work page 2018
[7]

Adaptive greedy approximations

Davis, G., Mallat, S., and Avellaneda, M. Adaptive greedy approximations. Constructive Approximation, 13: 0 57--98, 1997

work page 1997
[8]

Donoho, D. L. Compressed sensing. IEEE Transactions on Information Theory, 52 0 (4): 0 1289--1306, 2006

work page 2006

Show all 40 references

[9]

and Brodley, C

Dy, J. and Brodley, C. Feature subset selection and order identification for unsupervised learning. Proceedings of the Seventeenth International Conference on Machine Learning, 2000

2000
[10]

Least angle regression

Efron, B., Hastie, T., Johnstone, I., and Tibshirani, R. Least angle regression. The Annals of Statistics, 32 0 (2): 0 407--451, 2004

2004
[11]

and Li, R

Fan, J. and Li, R. Variable selection via nonconcave penalized likelihood and its oracle properties. Journal of the American Statistical Association, 96 0 (456): 0 1348--1360, 2001

2001
[12]

A threshold of ln n for approximating set cover

Feige, U. A threshold of ln n for approximating set cover. Journal of the ACM (JACM), 45 0 (4): 0 634--652, 1998

1998
[13]

Hard thresholding pursuit: An algorithm for compressive sensing

Foucart, S. Hard thresholding pursuit: An algorithm for compressive sensing. SIAM Journal on Numerical Analysis, 49 0 (6): 0 2543--2563, 2011

2011
[14]

F., Ellis, D

Gemmeke, J. F., Ellis, D. P., Freedman, D., Jansen, A., Lawrence, W., Moore, R. C., Plakal, M., and Ritter, M. Audio set: An ontology and human-labeled dataset for audio events. In 2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp.\ 776...

2017
[15]

Superconductivty Data

Hamidieh, K. Superconductivty Data . UCI Machine Learning Repository, 2018. DOI : https://doi.org/10.24432/C53P47

2018 doi
[16]

The elements of statistical learning: data mining, inference, and prediction, 2017

Hastie, T., Tibshirani, R., and Friedman, J. The elements of statistical learning: data mining, inference, and prediction, 2017

2017
[17]

and Mazumder, R

Hazimeh, H. and Mazumder, R. Fast best subset selection: Coordinate descent and local combinatorial optimization algorithms. Operations Research, 68 0 (5): 0 1517--1537, 2020

2020
[18]

Learning with structured sparsity

Huang, J., Zhang, T., and Metaxas, D. Learning with structured sparsity. In Proceedings of the 26th Annual International Conference on Machine Learning, pp.\ 417--424, 2009

2009
[19]

and John, G

Kohavi, R. and John, G. H. Wrappers for feature subset selection. Artificial Intelligence, 97 0 (1-2): 0 273--324, 1997

1997
[20]

and Pan, W

Lin, Z. and Pan, W. A robust cis-mendelian randomization method with application to drug target discovery. Nature Communications, 15 0 (1): 0 6072, 2024

2024
[21]

Mallat, S. G. and Zhang, Z. Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing, 41 0 (12): 0 3397--3415, 1993

1993
[22]

Newtonized orthogonal matching pursuit: Frequency estimation over the continuum

Mamandipoor, B., Ramasamy, D., and Madhow, U. Newtonized orthogonal matching pursuit: Frequency estimation over the continuum. IEEE Transactions on Signal Processing, 64 0 (19): 0 5066--5081, 2016

2016
[23]

Subset selection in regression

Miller, A. Subset selection in regression. Chapman and Hall/CRC, 2002

2002
[24]

and Tropp, J

Needell, D. and Tropp, J. Cosamp: Iterative signal recovery from incomplete and inaccurate samples. Applied and Computational Harmonic Analysis, 26 0 (3): 0 301--321, 2009

2009
[25]

C., Rezaiifar, R., and Krishnaprasad, P

Pati, Y. C., Rezaiifar, R., and Krishnaprasad, P. S. Orthogonal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In Proceedings of 27th Asilomar Conference on Signals, Systems and Computers, pp.\ 40--44. IEEE, 1993

1993
[26]

Scikit-learn: Machine learning in P ython

Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., and Duchesnay, E. Scikit-learn: Machine learning in P ython. Journal of ...

2011
[27]

Subset selection by pareto optimization

Qian, C., Yu, Y., and Zhou, Z.-H. Subset selection by pareto optimization. Advances in Neural Information Processing Systems, 28, 2015

2015
[28]

Subset selection under noise

Qian, C., Shi, J.-C., Yu, Y., Tang, K., and Zhou, Z.-H. Subset selection under noise. Advances in Neural Information Processing Systems, 30, 2017

2017
[29]

Statistics and Machine Learning Toolbox, 2025

The MathWorks, Inc. Statistics and Machine Learning Toolbox, 2025

2025
[30]

Regression shrinkage and selection via the lasso

Tibshirani, R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology, 58 0 (1): 0 267--288, 1996

1996
[31]

Revisiting matching pursuit: Beyond approximate submodularity

Tohidi, E., Coutino, M., and Gesbert, D. Revisiting matching pursuit: Beyond approximate submodularity. Signal Processing, 226: 0 109638, 2025

2025
[32]

Tropp, J. A. and Gilbert, A. C. Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory, 53 0 (12): 0 4655--4666, 2007

2007
[33]

OpenML Dataset 574: House 16H , 2014

Vanschoren, J. OpenML Dataset 574: House 16H , 2014

2014
[34]

A survey on data selection for LLM instruction tuning

Wang, J., Zhang, B., Du, Q., Zhang, J., and Chu, D. A survey on data selection for LLM instruction tuning. arXiv preprint arXiv:2402.05123, 2024

2024 arXiv
[35]

Submodularity in data subset selection and active learning

Wei, K., Iyer, R., and Bilmes, J. Submodularity in data subset selection and active learning. In Proceedings of the 32nd International Conference on Machine Learning, pp.\ 1954--1963. PMLR, 2015

1954
[36]

Nearly unbiased variable selection under minimax concave penalty

Zhang, C.-H. Nearly unbiased variable selection under minimax concave penalty . The Annals of Statistics, 38 0 (2): 0 894 -- 942, 2010

2010
[37]

A polynomial algorithm for best-subset selection problem

Zhu, J., Wen, C., Zhu, J., Zhang, H., and Wang, X. A polynomial algorithm for best-subset selection problem. Proceedings of the National Academy of Sciences, 117 0 (52): 0 33117--33123, 2020

2020
[38]

abess: A fast best-subset selection library in python and R

Zhu, J., Wang, X., Hu, L., Huang, J., Jiang, K., Zhang, Y., Lin, S., and Zhu, J. abess: A fast best-subset selection library in python and R . Journal of Machine Learning Research, 23 0 (202): 0 1--7, 2022

2022
[39]

The adaptive lasso and its oracle properties

Zou, H. The adaptive lasso and its oracle properties. Journal of the American Statistical Association, 101 0 (476): 0 1418--1429, 2006

2006
[40]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

A determinantal point process for column subset selection

Belhadji, A., Bardenet, R., and Chainais, P. A determinantal point process for column subset selection. Journal of Machine Learning Research, 21 0 (197): 0 1--62, 2020

work page 2020

[2] [2]

and Davies, M

Blumensath, T. and Davies, M. E. Gradient pursuits. IEEE Transactions on Signal Processing, 56 0 (6): 0 2370--2382, 2008

work page 2008

[3] [3]

and Davies, M

Blumensath, T. and Davies, M. E. Iterative hard thresholding for compressed sensing. Applied and Computational Harmonic Analysis, 27 0 (3): 0 265--274, 2009

work page 2009

[4] [4]

S., Donoho, D

Chen, S. S., Donoho, D. L., and Saunders, M. A. Atomic decomposition by basis pursuit. SIAM Review, 43 0 (1): 0 129--159, 2001

work page 2001

[5] [5]

and Kempe, D

Das, A. and Kempe, D. Submodular meets spectral: greedy algorithms for subset selection, sparse approximation and dictionary selection. In Proceedings of the 28th International Conference on International Conference on Machine Learning, pp.\ 1057–1064, 2011

work page 2011

[6] [6]

and Kempe, D

Das, A. and Kempe, D. Approximate submodularity and its applications: Subset selection, sparse approximation and dictionary selection. Journal of Machine Learning Research, 19 0 (3): 0 1--34, 2018

work page 2018

[7] [7]

Adaptive greedy approximations

Davis, G., Mallat, S., and Avellaneda, M. Adaptive greedy approximations. Constructive Approximation, 13: 0 57--98, 1997

work page 1997

[8] [8]

Donoho, D. L. Compressed sensing. IEEE Transactions on Information Theory, 52 0 (4): 0 1289--1306, 2006

work page 2006