REVIEW 4 major objections 6 minor 1 cited by
Enhancing Unsupervised Feature Selection via Double Sparsity Constrained Optimization
T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash
Pith's one-line read This paper claims that adding an ℓ0 element-wise sparsity constraint on top of an ℓ2,0 row-sparsity constraint in a PCA-based unsupervised feature selection model yields more discriminative features and better clustering than…
desk verdict The double-sparsity PCA model is new and the experiments are honest, but the global convergence theorem has a load-bearing proof gap and the label-tuned evaluation overclaims significance. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the penalty problem (7), where the orthogonal constraint is kept on $X$ while auxiliary variables $Y$ and $Z$ carry the two sparsity constraints and are tied to $X$ by quadratic penalties. The $X$-update is solved on a ball constraint using an exact penalty function method (Algorithm 2), and the $Y$- and $Z$-updates are projections onto the entry-sparse and row-sparse sets, implemented by hard thresholding operators. The convergence argument rests on the descent inequality (32) and the KL property of the semi-algebraic objective, which together are claimed to give global convergence to a stationary point.
What would settle it
Take a data matrix $A$ with nonzero covariance and run Algorithm 1 from $X_0=Y_0=Z_0=tI$; along this ray $f = -t^2\operatorname{Tr}(AA^{\top})$, which goes to $-\infty$ as $t$ grows, so the boundedness proof that invokes coercivity cannot hold. One can also check whether the $X$-updates returned by Algorithm 2 satisfy $X^{\top}X \approx I_m$ to the claimed tolerance; if they do not, the exact-minimizer premise of the descent inequality is violated.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is that $\|X\|_{2,0}\le r$ and $\|X\|_{0}\le s$ are complementary: row sparsity removes irrelevant and redundant features, while entry sparsity removes irregular local noise, and together they make the selected features more discriminative than either sparsity alone. The paper demonstrates this by adding the two constraints to the PCA model $\min \,-\operatorname{Tr}(X^{\top}AA^{\top}X)$ with $X^{\top}X = I_m$, solving the penalty reformulation by alternating updates, and reporting that DSCOFS is the only method in the comparison that beats the all-features baseline on every real dataset. The theoretical claim is that the iterates globally converge to a stationary point of the penalty problem, with the objective strictly nonincreasing, via the Kurdyka-Lojasiewicz property.
Load-bearing premise
The convergence guarantee assumes the penalty objective is coercive—that $f(X,Y,Z)$ grows to infinity when the variables grow—and that each $X$-subproblem is solved by an exact global minimizer; the empirical gains also assume it is legitimate to tune the number of selected features and sparsity parameters on the true cluster labels.
Editorial extensions
If this is right
- On all eight real-world datasets tested, DSCOFS achieves the best or second-best ACC and NMI among the compared methods, averaging gains of at least 3.34% in ACC and 3.02% in NMI.
- On the three synthetic datasets, only DSCOFS selects the two correct original features, suggesting double sparsity recovers relevant features that single-sparsity methods miss.
- The feature similarity ratio shows that adding $\ell_0$-sparsity changes the selected feature set substantially (e.g., 52% overlap on Isolet), so the improvement is not just a re-ranking of the same features.
- Statistical tests (Friedman and post-hoc Nemenyi) reject the null hypothesis of equal performance and find DSCOFS significantly different from several single-sparsity baselines.
- The claimed convergence results imply the alternating scheme is a trustworthy solver for the penalty problem: the objective decreases monotonically and every limit point is stationary.
Reading between the lines
- Editorial inference: If double sparsity is the source of the gain, then on new datasets with noisy high-dimensional features one would expect the optimal element-wise sparsity percentage to be low (around $\alpha=0.1$), because the parameter sensitivity plots show the largest improvements there.
- Editorial inference: The same row-plus-entry sparsity mechanism could be applied to other unsupervised objectives such as matrix factorization or canonical correlation analysis, since the hard-thresholding updates only require a quadratic least-squares structure.
- Editorial inference: The feature similarity ratio could serve as a general diagnostic for any two-stage feature-selection method, measuring how much an additional constraint changes the chosen feature set before evaluating downstream accuracy.
- Editorial inference: The comparison with a deep teacher-student method, while limited to four datasets, suggests the hypothesis that double sparsity captures structure that data-hungry deep selectors miss on small samples; a larger benchmark would test whether the gap persists.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an unsupervised feature-selection method, DSCOFS, that adds an element-wise ℓ0-norm constraint to the row-sparse ℓ2,0-norm PCA model, giving a double-sparsity constrained problem. The authors reformulate the model with auxiliary variables, solve the subproblems by a proximal alternating minimization scheme (with an exact-penalty inner solver for the orthogonality-constrained X-subproblem and hard-thresholding for Y and Z), and claim a global convergence theorem: the sequence generated by Algorithm 1 monotonically decreases the objective, is bounded, and converges to a stationary point of the penalty problem. Experiments on three synthetic and eight real-world datasets report average ACC and NMI improvements of at least 3.34% and 3.02% over several existing UFS methods, together with an ablation study, statistical tests, and a new feature-similarity metric (FSR).
Significance. If the convergence theorem were correct, it would be a valuable theoretical guarantee for a nonconvex nonsmooth sparsity-constrained PCA model, and the double-sparsity formulation is a natural and potentially useful extension of single-sparsity UFS. The algorithm is computationally attractive because the Y- and Z-subproblems have closed-form hard-thresholding solutions, and the experimental section is extensive: it includes multiple baselines, ablation of the ℓ0 constraint, statistical tests, a stability analysis, and a comparison with a deep UFS method. These are real strengths. However, the advertised global convergence result is the paper's central theoretical contribution, and the proof as written contains load-bearing gaps: the descent inequality assumes an exact global solution that the proposed algorithm does not compute, and the coercivity/bounded-below argument for f is false as stated. Because the headline claim is not established, the theoretical significance of the paper in its current form is much reduced.
major comments (4)
- [Section 3.3, Eq. (31) and Algorithm 2] The descent inequality (31) is valid only if X^{k+1} is an exact global minimizer of subproblem (8) and hence is feasible for the orthogonal manifold M. Algorithm 2 does not solve (8): it replaces (8) by the penalty surrogate (11) on the ball B_ρ and generates only approximate iterates via (16)-(19), whose output need not be in M. The equivalence quoted from [35] concerns global minimizers, not the approximate stationary points actually produced by Algorithm 2. Consequently, the monotonicity asserted in Theorem 3.2(a), and everything built on it, is not established by the given proof.
- [Section 3.3, Theorem 3.2(b)-(c) and Appendix proof] The proof of Theorem 3.2(b) asserts that f in (24) is coercive and concludes from unboundedness of the sequence that f diverges to +∞. This is false on the stated domain: taking X=Y=Z=tI gives f(X,Y,Z) = -t^2 Tr(AA^T), which tends to -∞ as t→∞ (whenever Tr(AA^T)>0). The proof of (c) also uses the claim that f is bounded below. Although boundedness might be salvageable by restricting to the feasible set M×S×R and using compactness of M together with the growth of the penalty terms, the proof as written does not do this. The boundedness and limit claims are therefore unsupported as stated.
- [Section 3.2.1 and Theorem 3.2(d), Eqs. (39)-(50)] The first-order optimality conditions (39) are invoked at X^{k+1} as if it were an exact solution of subproblem (8), but Algorithm 2 provides no such solution. In addition, the passage from a stationary point of the penalty problem (11) to a stationary point of (8) via [35, Theorem 3.1] requires the iterate to be an actual stationary point of (11); Algorithm 2 is run only until a stopping criterion and does not guarantee stationarity of (11), let alone global optimality of (8). Thus the concluding claim that every accumulation point is a stationary point of (7) does not follow from the arguments presented.
- [Section 4.1.2 and Tables 2-3] The experimental protocol selects regularization parameters and the number of selected features by grid search using the best ACC/NMI values computed against the true cluster labels. This is a supervised model-selection procedure inside an unsupervised feature-selection pipeline, and it can systematically inflate the reported gains, including the headline average improvements of 3.34% (ACC) and 3.02% (NMI). The comparison should be repeated, or at least supplemented, with a protocol that does not use ground-truth labels for parameter selection, and the current protocol should be disclosed as such.
minor comments (6)
- [Section 3.2.1, Algorithm 2] The symbol X^{k+1} is reused both for the inner iterates of Algorithm 2 and for the outer iterate returned to Algorithm 1; this makes the algorithm difficult to follow and should be clarified with a different inner-index notation.
- [Section 4.2.2, Table 2] In the COIL20 row, the parentheses around the number of selected features are unbalanced: "100)" should read "(100)".
- [Section 4.3.1, Eq. (30)] The symbol n denotes the number of samples in Section 2.1 but denotes the number of selected features in the definition of FSR; a distinct symbol should be used for the feature count.
- [Section 3.2, Algorithm 1 input] There is a typo in the algorithm caption: "paraments" should be "parameters".
- [Section 3.3, Theorem 3.2(a) and contribution list] The decrease is described as "strictly nonincreasing," but the proof establishes nonincrease; strict decrease would require additional argument that consecutive iterates differ. The wording should be made consistent.
- [Section 4.4.3] The empirical convergence curves show that the objective decreases on four datasets, but they do not verify the claimed global convergence to a stationary point; the text should not present these plots as confirmation of Theorem 3.2.
Circularity Check
No significant circularity; theoretical and empirical claims are not equivalent to their inputs by construction.
full rationale
The paper's empirical claims (ACC/NMI improvements over LapScore, UDFS, SOGFS, RNE, FSPCA, SPCAFS, and SPCA-PSD) are benchmarked against external baselines with shared hyperparameter-search protocols, so they do not reduce by construction to fitted constants. The convergence proof is an internal mathematical argument that invokes external theorems ([35], [36], [39]) with no author overlap with the present paper, and the only self-citation ([33]) is motivational rather than load-bearing. The FSR metric is definitionally a measure of overlap between the method's double-sparse and single-sparse outputs; its use as descriptive evidence of different feature selections is not a circular prediction. Concerns about Theorem 3.2 (inequality (31) requiring exact global minimizers of (8) while Algorithm 2 returns approximate stationary points of the penalized problem (11), and the imprecise coercivity statement in the proof of part (b)) are correctness risks or proof gaps, not circularity, because the claimed result is not obtained by assuming its conclusion or by defining one quantity in terms of another. No specific reduction from output back to input can be exhibited from the text, so the appropriate finding is no significant circularity.
Assumptions & free parameters
free parameters (5)
- r (row sparsity level) =
varies per dataset, e.g., 100 for Isolet, 40 for warpPIE10P (Table 2)
- α (element-wise sparsity percentage) =
chosen from {0.1,...,0.9} per dataset; values not reported in tables
- μ1, μ2 (penalty parameters) =
not reported
- τ1, τ2, τ3 (proximal parameters) =
not reported
- β, ρ, η (exact penalty parameters) =
not reported
assumptions (4)
- ad hoc to paper f(X,Y,Z) is coercive and bounded below
- domain assumption The exact penalty function method [35, Thm 3.2] makes (11) equivalent to (8)
- standard math Subproblems (9) and (10) are solved globally by hard thresholding
- standard math The KL property holds for the objective with indicator functions of M, S and R
invented entities (1)
-
Feature similarity ratio (FSR)
Cite this review
Pith. "Pith review of Enhancing Unsupervised Feature Selection via Double Sparsity Constrained Optimization." pith.science (2026). https://pith.science/paper/ZV562JTH
@misc{pith2026250100726,
author = {Pith},
title = {Pith review of: Enhancing Unsupervised Feature Selection via Double Sparsity Constrained Optimization},
year = {2026},
howpublished = {\url{https://pith.science/paper/ZV562JTH}},
note = {Machine review of arXiv:2501.00726}
}
abstract
Unsupervised feature selection (UFS) is widely applied in machine learning and pattern recognition. However, most of the existing methods only consider a single sparsity, which makes it difficult to select valuable and discriminative feature subsets from the original high-dimensional feature set. In this paper, we propose a new UFS method called DSCOFS via embedding double sparsity constrained optimization into the classical principal component analysis (PCA) framework. Double sparsity refers to using $\ell_{2,0}$-norm and $\ell_0$-norm to simultaneously constrain variables, by adding the sparsity of different types, to achieve the purpose of improving the accuracy of identifying differential features. The core is that $\ell_{2,0}$-norm can remove irrelevant and redundant features, while $\ell_0$-norm can filter out irregular noisy features, thereby complementing $\ell_{2,0}$-norm to improve discrimination. An effective proximal alternating minimization method is proposed to solve the resulting nonconvex nonsmooth model. Theoretically, we rigorously prove that the sequence generated by our method globally converges to a stationary point. Numerical experiments on three synthetic datasets and eight real-world datasets demonstrate the effectiveness, stability, and convergence of the proposed method. In particular, the average clustering accuracy (ACC) and normalized mutual information (NMI) are improved by at least 3.34% and 3.02%, respectively, compared with the state-of-the-art methods. More importantly, two common statistical tests and a new feature similarity metric verify the advantages of double sparsity. All results suggest that our proposed DSCOFS provides a new perspective for feature selection.
Figures
Figures from the paper (8 more)
Forward citations
Cited by 1 Pith paper
-
Variance-Preserving Orthogonal Selection (VPOS): Greedy Feature Selection via Orthogonal Deflation in PCA Loading Space
VPOS greedily selects the feature with the largest weighted PCA loading norm, deflates that direction, and reports the lowest reconstruction MSE on eight benchmarks under a minimum-MSE d-selection rule.
Reference graph
Works this paper leans on
-
[35]
N. Xiao, X. Liu, Y . Yuan, A class of smooth exact penalty function methods for optimization problems with orthogonality constraints, Optimization Methods and Software 37 (4) (2022) 1205–1241
work page 2022
-
[1]
Y . Fan, J. Liu, J. Tang, P. Liu, Y . Lin, Y . Du, Learning correlation information for multi-label feature selection, Pattern Recognition 145 (2024) 109899
work page 2024
-
[2]
Q. Zhou, Q. Wang, Q. Gao, M. Yang, X. Gao, Unsupervised discriminative fea- ture selection via contrastive graph learning, IEEE Transactions on Image Pro- cessing 33 (2024) 972–986
work page 2024
- [3]
-
[4]
M. P. Uddin, M. A. Mamun, M. I. Afjal, M. A. Hossain, Information-theoretic feature selection with segmentation-based folded principal component analysis (PCA) for hyperspectral image classification, International Journal of Remote Sensing 42 (1) (2021) 286–321. 31
work page 2021
-
[5]
Z. Liu, Y . Yang, F. Gao, T. Zhou, H. Ma, Deep unsupervised learning for joint an- tenna selection and hybrid beamforming, IEEE Transactions on Communications 70 (3) (2022) 1697–1710
work page 2022
-
[6]
F. Saberi-Movahed, M. Rostami, K. Berahmand, S. Karami, P. Tiwari, M. Ous- salah, S. S. Band, Dual regularized unsupervised feature selection based on ma- trix factorization and minimum redundancy with application in gene selection, Knowledge-Based Systems 256 (2022) 109884
work page 2022
-
[7]
P. Dhal, C. Azad, A comprehensive survey on feature selection in the various fields of machine learning, Applied Intelligence 52 (4) (2022) 4543–4581
work page 2022
Show all 40 references
-
[8]
G. Li, Z. Yu, K. Yang, M. Lin, C. L. P. Chen, Exploring feature selection with limited labels: A comprehensive survey of semi-supervised and unsupervised ap- proaches, IEEE Transactions on Knowledge and Data Engineering 36 (11) (2024) 6124–6144
2024
-
[9]
X. He, D. Cai, P. Niyogi, Laplacian score for feature selection, Advances in Neu- ral Information Processing Systems 18 (2005)
2005
-
[10]
Y . Yang, H. T. Shen, Z. Ma, Z. Huang, X. Zhou, ℓ2,1-norm regularized discrim- inative feature selection for unsupervised learning, in: IJCAI International Joint Conference on Artificial Intelligence, 2011
2011
-
[11]
F. Nie, W. Zhu, X. Li, Structured graph optimization for unsupervised feature selection, IEEE Transactions on Knowledge and Data Engineering 33 (3) (2021) 1210–1222
2021
-
[12]
Y . Liu, D. Ye, W. Li, H. Wang, Y . Gao, Robust neighborhood embedding for unsupervised feature selection, Knowledge-Based Systems 193 (2020) 105462
2020
-
[13]
H. Zou, T. Hastie, R. Tibshirani, Sparse principal component analysis, Journal of Computational and Graphical Statistics 15 (2) (2006) 265–286
2006
-
[14]
Chang, F
X. Chang, F. Nie, Y . Yang, C. Zhang, H. Huang, Convex sparse PCA for unsuper- vised feature learning, ACM Transactions on Knowledge Discovery from Data 11 (1) (2016) 1–16. 32
2016
-
[15]
S. Yi, Z. He, X. Jing, Y . Li, Y . Cheung, F. Nie, Adaptive weighted sparse principal component analysis for robust unsupervised feature selection, IEEE Transactions on Neural Networks and Learning Systems 31 (6) (2020) 2153–2163
2020
-
[16]
Z. Li, F. Nie, J. Bian, D. Wu, X. Li, Sparse PCA via ℓ2,p-norm regularization for unsupervised feature selection, IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (4) (2023) 5322–5328
2023
-
[17]
Zheng, X
J. Zheng, X. Zhang, Y . Liu, W. Jiang, K. Huo, L. Liu, Fast sparse PCA via posi- tive semidefinite projection for unsupervised feature selection, arXiv: 2309.06202 (Available: https://arxiv.org/abs/2309.06202)
-
[18]
T. Pang, F. Nie, J. Han, X. Li, Efficient feature selection viaℓ2,0-norm constrained sparse regression, IEEE Transactions on Knowledge and Data Engineering 31 (5) (2019) 880–893
2019
-
[19]
J. Wang, H. Wang, F. Nie, X. Li, Sparse feature selection via fast embedding spectral analysis, Pattern Recognition 139 (2023) 109472
2023
-
[20]
H. Chen, F. Nie, R. Wang, X. Li, Fast unsupervised feature selection with bipar- tite graph and ℓ2,0-norm constraint, IEEE Transactions on Knowledge and Data Engineering 35 (5) (2023) 4781–4793
2023
-
[21]
F. Nie, L. Tian, R. Wang, X. Li, Learning feature-sparse principal subspace, IEEE Transactions on Pattern Analysis and Machine Intelligence 45 (4) (2023) 4858– 4869
2023
-
[22]
L. Cui, L. Bai, Y . Wang, S. Y . Philip, E. R. Hancock, Fused lasso for feature selection using structural information, Pattern Recognition 119 (2021) 108058
2021
-
[23]
J. Liu, M. Feng, X. Xiu, W. Liu, Towards robust and sparse linear discriminant analysis for image classification, Pattern Recognition 153 (2024) 110512
2024
-
[24]
X. Bian, W. Xu, Y . Wang, Z. Cai, C. Yuen, Joint compressed signal recovery and ris diagnosis via double-sparsity optimization, IEEE Internet of Things Journal 11 (8) (2024) 13327–13339. 33
2024
-
[25]
Zhang, Y
S. Zhang, Y . Liu, X. Li, Micro-doppler effects removed sparse aperture isar imag- ing via low-rank and double sparsity constrained admm and linearized admm, IEEE Transactions on Image Processing 30 (2021) 4678–4690
2021
-
[26]
S. Zhou, Z. Luo, N. Xiu, G. Y . Li, Computing one-bit compressive sensing via double-sparsity constrained optimization, IEEE Transactions on Signal Process- ing 70 (2022) 1593–1608
2022
-
[27]
Y . Guo, Y . Sun, Z. Wang, F. Nie, F. Wang, Double-structured sparsity guided flex- ible embedding learning for unsupervised feature selection, IEEE Transactions on Neural Networks and Learning Systems 35 (10) (2024) 13354–13367
2024
-
[28]
H. Wang, F. Nie, H. Huang, S. Risacher, C. Ding, A. J. Saykin, L. Shen, Sparse multi-task regression and feature selection to identify brain imaging predictors for memory performance, in: 2011 International Conference on Computer Vision, IEEE, 2011, pp. 557–562
2011
-
[29]
Y . Hu, J. Liu, Y . Gao, J. Shang, DSTPCA: Double-sparse constrained tensor prin- cipal component analysis method for feature selection, IEEE /ACM Transactions on Computational Biology and Bioinformatics 18 (4) (2021) 1481–1491
2021
-
[30]
J. Sun, L. Kong, S. Zhou, Gradient projection Newton algorithm for sparse col- laborative learning using synthetic and real datasets of applications, Journal of Computational and Applied Mathematics 422 (2023) 114872
2023
-
[31]
F. Nie, Q. Chen, W. Yu, X. Li, Row-sparse principal component analysis via coor- dinate descent method, IEEE Transactions on Knowledge and Data Engineering 36 (7) (2024) 3460–3471
2024
-
[32]
Z. Wang, Q. Li, H. Zhao, F. Nie, Simultaneous local clustering and unsupervised feature selection via strong space constraint, Pattern Recognition 142 (2023) 109718
2023
-
[33]
X. Xiu, L. Pan, Y . Yang, W. Liu, Efficient and fast joint sparse constrained canon- ical correlation analysis for fault detection, IEEE Transactions on Neural Net- works and Learning Systems 35 (3) (2024) 4153–4163. 34
2024
-
[34]
S. Chen, S. Ma, L. Xue, H. Zou, An alternating manifold proximal gradient method for sparse principal component analysis and sparse canonical correlation analysis, INFORMS Journal on Optimization 2 (3) (2020) 192–208
2020
-
[36]
B. Gao, X. Liu, X. Chen, Y . Yuan, A new first-order algorithmic framework for optimization problems with orthogonality constraints, SIAM Journal on Opti- mization 28 (1) (2018) 302–332
2018
-
[37]
Huang, Y
Y . Huang, Y . Dai, X. Liu, Equipping the Barzilai–Borwein method with the two dimensional quadratic termination property, SIAM Journal on Optimization 31 (4) (2021) 3068–3096
2021
-
[38]
Blumensath, M
T. Blumensath, M. E. Davies, Iterative hard thresholding for compressed sensing, Applied and Computational Harmonic Analysis 27 (3) (2009) 265–274
2009
-
[39]
Bolte, S
J. Bolte, S. Sabach, M. Teboulle, Proximal alternating linearized minimization for nonconvex and nonsmooth problems, Mathematical Programming 146 (1) (2014) 459–494
2014
-
[40]
Mirzaei, V
A. Mirzaei, V . Pourahmadi, M. Soltani, H. Sheikhzadeh, Deep feature selection using a teacher-student network, Neurocomputing 383 (2020) 396–408. 35
2020
Reviewed August 10, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.