{"id":"d37f07b0-63a4-426e-823a-2bb535ff4233","arxiv_id":"2501.00726","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"DSCOFS embeds simultaneous ℓ2,0 row sparsity and ℓ0 element sparsity into PCA for unsupervised feature selection, with an alternating minimization solver and claimed gains in clustering accuracy and normalized mutual information.","lead":"This paper proposes DSCOFS, an unsupervised feature selection method that adds two sparsity constraints, one on feature rows and one on individual matrix entries, to a PCA objective. It reports consistent clustering accuracy gains over seven baselines on eight real-world datasets, though the proof of the method's convergence contains a load-bearing error.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Global convergence proof fails because Algorithm 2 does not produce the exact global minimizer of (8) required by inequality (31).","rationale":"The reader's verdict identifies both the coercivity error and the exact-minimization gap in the proof of Theorem 3.2. I agree that the exact-minimization gap is load-bearing and that the convergence proof is invalid as written. I do not think the coercivity issue is equally load-bearing because, on the actual constraint set M×S×R, the objective f is likely coercive: X is restricted to the compact Stiefel manifold, and the penalty terms μ1||X-Y||^2 + μ2||X-Z||^2 grow quadratically when Y or Z become unbounded. Thus the proof of boundedness could be repaired by restricting the coercivity claim to the feasible set. By contrast, the exact-minimization gap cannot be repaired without either changing Algorithm 2 to compute a true global minimizer of (8), which is intractable in general, or proving an inexact sufficient-decrease condition that the approximate gradient-descent update actually satisfies. No such condition is stated or proved. The convergence analysis is a headline contribution of the paper, and its failure is sufficient grounds for rejection, despite the fact that the empirical method may still work in practice. The proposed concrete test would settle the issue by checking the descent inequality on a small instance, and the test is inexpensive to run.","tokens_in":22863,"tokens_out":3815,"duration_ms":37976,"concrete_test":"Run Algorithm 1 on a small synthetic instance (e.g., d=5, m=2, n=20, random A) with the X-update computed in two ways: (i) by an exact solver that globally minimizes (8) over the Stiefel manifold M (for small m,d this can be done by enumeration over a sufficiently fine grid or by branch-and-bound), and (ii) by Algorithm 2 exactly as described in the paper. At each iteration, directly check whether the inequality f(X^{k+1},Y^k,Z^k) + τ1||X^{k+1}-X^k||^2 ≤ f(X^k,Y^k,Z^k) holds. If Algorithm 2's output violates this inequality at any iteration, then the premise of Theorem 3.2(a) is not satisfied by the implemented method. Additionally, record ||(X^{k+1})^T X^{k+1} - I_m||_F at the final iterate; if it is not close to zero, the final point is not even feasible for the original orthogonality constraint, so it cannot serve as a stationary point of (7).","verdict_should_be":"REJECT","load_bearing_attack":"The central theoretical claim is Theorem 3.2: the sequence generated by Algorithm 1 globally converges to a stationary point of (7). Its proof begins with inequality (31), which is valid only if X^{k+1} is an exact global minimizer of the X-subproblem (8): the inequality f(X^{k+1},Y^k,Z^k) + τ1||X^{k+1}-X^k||^2 ≤ f(X^k,Y^k,Z^k) uses the fact that X^k is feasible for (8) and X^{k+1} is a global minimizer. However, Algorithm 1 calls Algorithm 2 to update X, and Algorithm 2 does not solve (8) exactly. It replaces (8) by the penalty problem (11) with the ball constraint C = B_ρ, then runs approximate gradient descent with a Barzilai-Borwein step and a projection onto B_ρ. At best it returns an approximate stationary point of the nonconvex problem (11), not a global minimizer of (8). The cited equivalence [35, Theorem 3.2] only concerns global minimizers, and Algorithm 2 provides no guarantee of reaching one. Consequently, inequality (31) is not established, the monotonicity in Theorem 3.2(a) is unsupported, and the KL-based convergence argument in Theorem 3.2(d) collapses. A secondary issue is that the proof of Theorem 3.2(b) asserts that f is coercive, which is false on the whole space since f(tI,tI,tI) -> -∞; while f may be coercive on the actual feasible set M×S×R (because M is bounded and the penalty terms grow quadratically in Y and Z), the written proof does not state or use this restricted coercivity, and the unbounded-below example does not satisfy the orthogonality constraint. Still, the exactness gap is the more load-bearing defect because it undermines the descent argument that both monotonicity and convergence rely on.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes an unsupervised feature-selection method, DSCOFS, that adds an element-wise ℓ0-norm constraint to the row-sparse ℓ2,0-norm PCA model, giving a double-sparsity constrained problem. The authors reformulate the model with auxiliary variables, solve the subproblems by a proximal alternating minimization scheme (with an exact-penalty inner solver for the orthogonality-constrained X-subproblem and hard-thresholding for Y and Z), and claim a global convergence theorem: the sequence generated by Algorithm 1 monotonically decreases the objective, is bounded, and converges to a stationary point of the penalty problem. Experiments on three synthetic and eight real-world datasets report average ACC and NMI improvements of at least 3.34% and 3.02% over several existing UFS methods, together with an ablation study, statistical tests, and a new feature-similarity metric (FSR).","tokens_in":23167,"tokens_out":6996,"duration_ms":70652,"significance":"If the convergence theorem were correct, it would be a valuable theoretical guarantee for a nonconvex nonsmooth sparsity-constrained PCA model, and the double-sparsity formulation is a natural and potentially useful extension of single-sparsity UFS. The algorithm is computationally attractive because the Y- and Z-subproblems have closed-form hard-thresholding solutions, and the experimental section is extensive: it includes multiple baselines, ablation of the ℓ0 constraint, statistical tests, a stability analysis, and a comparison with a deep UFS method. These are real strengths. However, the advertised global convergence result is the paper's central theoretical contribution, and the proof as written contains load-bearing gaps: the descent inequality assumes an exact global solution that the proposed algorithm does not compute, and the coercivity/bounded-below argument for f is false as stated. Because the headline claim is not established, the theoretical significance of the paper in its current form is much reduced.","major_comments":[{"comment":"The descent inequality (31) is valid only if X^{k+1} is an exact global minimizer of subproblem (8) and hence is feasible for the orthogonal manifold M. Algorithm 2 does not solve (8): it replaces (8) by the penalty surrogate (11) on the ball B_ρ and generates only approximate iterates via (16)-(19), whose output need not be in M. The equivalence quoted from [35] concerns global minimizers, not the approximate stationary points actually produced by Algorithm 2. Consequently, the monotonicity asserted in Theorem 3.2(a), and everything built on it, is not established by the given proof.","section":"Section 3.3, Eq. (31) and Algorithm 2"},{"comment":"The proof of Theorem 3.2(b) asserts that f in (24) is coercive and concludes from unboundedness of the sequence that f diverges to +∞. This is false on the stated domain: taking X=Y=Z=tI gives f(X,Y,Z) = -t^2 Tr(AA^T), which tends to -∞ as t→∞ (whenever Tr(AA^T)>0). The proof of (c) also uses the claim that f is bounded below. Although boundedness might be salvageable by restricting to the feasible set M×S×R and using compactness of M together with the growth of the penalty terms, the proof as written does not do this. The boundedness and limit claims are therefore unsupported as stated.","section":"Section 3.3, Theorem 3.2(b)-(c) and Appendix proof"},{"comment":"The first-order optimality conditions (39) are invoked at X^{k+1} as if it were an exact solution of subproblem (8), but Algorithm 2 provides no such solution. In addition, the passage from a stationary point of the penalty problem (11) to a stationary point of (8) via [35, Theorem 3.1] requires the iterate to be an actual stationary point of (11); Algorithm 2 is run only until a stopping criterion and does not guarantee stationarity of (11), let alone global optimality of (8). Thus the concluding claim that every accumulation point is a stationary point of (7) does not follow from the arguments presented.","section":"Section 3.2.1 and Theorem 3.2(d), Eqs. (39)-(50)"},{"comment":"The experimental protocol selects regularization parameters and the number of selected features by grid search using the best ACC/NMI values computed against the true cluster labels. This is a supervised model-selection procedure inside an unsupervised feature-selection pipeline, and it can systematically inflate the reported gains, including the headline average improvements of 3.34% (ACC) and 3.02% (NMI). The comparison should be repeated, or at least supplemented, with a protocol that does not use ground-truth labels for parameter selection, and the current protocol should be disclosed as such.","section":"Section 4.1.2 and Tables 2-3"}],"minor_comments":[{"comment":"The symbol X^{k+1} is reused both for the inner iterates of Algorithm 2 and for the outer iterate returned to Algorithm 1; this makes the algorithm difficult to follow and should be clarified with a different inner-index notation.","section":"Section 3.2.1, Algorithm 2"},{"comment":"In the COIL20 row, the parentheses around the number of selected features are unbalanced: \"100)\" should read \"(100)\".","section":"Section 4.2.2, Table 2"},{"comment":"The symbol n denotes the number of samples in Section 2.1 but denotes the number of selected features in the definition of FSR; a distinct symbol should be used for the feature count.","section":"Section 4.3.1, Eq. (30)"},{"comment":"There is a typo in the algorithm caption: \"paraments\" should be \"parameters\".","section":"Section 3.2, Algorithm 1 input"},{"comment":"The decrease is described as \"strictly nonincreasing,\" but the proof establishes nonincrease; strict decrease would require additional argument that consecutive iterates differ. The wording should be made consistent.","section":"Section 3.3, Theorem 3.2(a) and contribution list"},{"comment":"The empirical convergence curves show that the objective decreases on four datasets, but they do not verify the claimed global convergence to a stationary point; the text should not present these plots as confirmation of Theorem 3.2.","section":"Section 4.4.3"}],"recommendation":"reject","confidential_remarks":"The central theoretical contribution of this manuscript, the global convergence theorem, is not established by the provided proof: the algorithm does not return the exact global minimizer required by the descent inequality, and the coercivity argument is based on a false statement about f. The experimental comparison is extensive, but the use of true labels for parameter selection inflates the reported improvements. In my view, preserving the advertised global convergence claim would require a substantially new convergence analysis rather than a local fix, so I recommend rejection in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know before you read this one. The DSCOFS formulation—ℓ2,0 and ℓ0 constraints on the same projection matrix inside a PCA objective—is genuinely new as far as the cited literature goes, and the ablation suggests element-wise sparsity changes which features get selected. The empirical gains over seven SPCA/spectral baselines are real but modest (about 3% ACC/NMI on average). The second thing is more important: the advertised global convergence proof does not hold as written.\n\nThe flaw is not cosmetic. Inequality (31) requires X^{k+1} to be a global minimizer of the X-subproblem (8). Algorithm 2 solves the penalty reformulation (11) with an approximate gradient descent and a projection onto a ball, and it returns at best an approximate stationary point—not a global minimizer. The cited equivalence [35, Thm 3.2] is only for global minimizers. So the descent guarantee, the boundedness proof (which also invokes a false coercivity of f on the whole space), and the KL-based convergence argument all lose their support. The method may still work in practice, but the paper's central theoretical claim is not established.\n\nOn the empirical side, the protocol tunes the number of features and all regularization parameters using the true cluster labels, so the reported ACC/NMI are in-sample optima. That inflates the numbers, though it inflates the baselines too, so the relative gains are not entirely artifactual. The Nemenyi test does not show DSCOFS significantly better than SPCA-PSD or SPCAFS, so the abstract's \"statistical tests verify the advantages\" is an overclaim.\n\nWhat's good: the model is clean and the algorithm has closed-form Y and Z updates, so it's a practical heuristic. The FSR metric is simple but does give a concrete way to see that ℓ0 sparsity changes feature rankings. The synthetic experiments are a nice sanity check.\n\nBottom line: this is a salvageable paper with a real formulation and a substantial experiment suite, but the main theorem is broken and the evaluation needs an honest validation split. A serious referee would send it back with major revision requests. I wouldn't cite it in its current form.","headline":"The double-sparsity PCA model is new and the experiments are honest, but the global convergence theorem has a load-bearing proof gap and the label-tuned evaluation overclaims significance.","tokens_in":23866,"tokens_out":4438,"would_cite":false,"duration_ms":42290,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["90C26","90C30","62H25","65K05"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that adding an ℓ0 element-wise sparsity constraint on top of an ℓ2,0 row-sparsity constraint in a PCA-based unsupervised feature selection model yields more discriminative features and better clustering than…","keywords":["unsupervised feature selection","sparse principal component analysis","double sparsity","ℓ2,0 norm","ℓ0 norm","proximal alternating minimization","global convergence","feature similarity ratio"],"falsifier":"Take a data matrix $A$ with nonzero covariance and run Algorithm 1 from $X_0=Y_0=Z_0=tI$; along this ray $f = -t^2\\operatorname{Tr}(AA^{\\top})$, which goes to $-\\infty$ as $t$ grows, so the boundedness proof that invokes coercivity cannot hold. One can also check whether the $X$-updates returned by Algorithm 2 satisfy $X^{\\top}X \\approx I_m$ to the claimed tolerance; if they do not, the exact-minimizer premise of the descent inequality is violated.","tokens_in":22550,"feed_emoji":"🎯","tokens_out":6543,"duration_ms":60757,"temperature":0.7,"pith_summary":"This paper tries to establish that unsupervised feature selection improves when the projection matrix is constrained by two sparsity norms at once: the ℓ2,0-norm, which forces whole rows (features) to zero, and the ℓ0-norm, which forces individual entries to zero. The proposed method, DSCOFS, embeds these constraints in the classical PCA objective and solves the resulting nonconvex problem with a proximal alternating minimization scheme whose subproblems have closed-form solutions or fast solvers. The paper claims this yields better discriminative features than single-sparsity PCA variants, with average clustering accuracy and normalized mutual information gains of at least 3.34% and 3.02% over comparison methods on eight real-world datasets. It also contributes a feature similarity ratio metric and a convergence analysis asserting global convergence to a stationary point.","feed_headline":"Double sparsity lifts unsupervised feature selection by 3%+","feed_subtitle":"A PCA model constrained by both ℓ2,0 and ℓ0 norms picks cleaner features and beats single-sparsity rivals on eight datasets.","key_machinery":"The load-bearing object is the penalty problem (7), where the orthogonal constraint is kept on $X$ while auxiliary variables $Y$ and $Z$ carry the two sparsity constraints and are tied to $X$ by quadratic penalties. The $X$-update is solved on a ball constraint using an exact penalty function method (Algorithm 2), and the $Y$- and $Z$-updates are projections onto the entry-sparse and row-sparse sets, implemented by hard thresholding operators. The convergence argument rests on the descent inequality (32) and the KL property of the semi-algebraic objective, which together are claimed to give global convergence to a stationary point.","core_discovery":"On the paper's own terms, the central discovery is that $\\|X\\|_{2,0}\\le r$ and $\\|X\\|_{0}\\le s$ are complementary: row sparsity removes irrelevant and redundant features, while entry sparsity removes irregular local noise, and together they make the selected features more discriminative than either sparsity alone. The paper demonstrates this by adding the two constraints to the PCA model $\\min \\,-\\operatorname{Tr}(X^{\\top}AA^{\\top}X)$ with $X^{\\top}X = I_m$, solving the penalty reformulation by alternating updates, and reporting that DSCOFS is the only method in the comparison that beats the all-features baseline on every real dataset. The theoretical claim is that the iterates globally converge to a stationary point of the penalty problem, with the objective strictly nonincreasing, via the Kurdyka-Lojasiewicz property.","pith_inferences":["Editorial inference: If double sparsity is the source of the gain, then on new datasets with noisy high-dimensional features one would expect the optimal element-wise sparsity percentage to be low (around $\\alpha=0.1$), because the parameter sensitivity plots show the largest improvements there.","Editorial inference: The same row-plus-entry sparsity mechanism could be applied to other unsupervised objectives such as matrix factorization or canonical correlation analysis, since the hard-thresholding updates only require a quadratic least-squares structure.","Editorial inference: The feature similarity ratio could serve as a general diagnostic for any two-stage feature-selection method, measuring how much an additional constraint changes the chosen feature set before evaluating downstream accuracy.","Editorial inference: The comparison with a deep teacher-student method, while limited to four datasets, suggests the hypothesis that double sparsity captures structure that data-hungry deep selectors miss on small samples; a larger benchmark would test whether the gap persists."],"forward_implications":["On all eight real-world datasets tested, DSCOFS achieves the best or second-best ACC and NMI among the compared methods, averaging gains of at least 3.34% in ACC and 3.02% in NMI.","On the three synthetic datasets, only DSCOFS selects the two correct original features, suggesting double sparsity recovers relevant features that single-sparsity methods miss.","The feature similarity ratio shows that adding $\\ell_0$-sparsity changes the selected feature set substantially (e.g., 52% overlap on Isolet), so the improvement is not just a re-ranking of the same features.","Statistical tests (Friedman and post-hoc Nemenyi) reject the null hypothesis of equal performance and find DSCOFS significantly different from several single-sparsity baselines.","The claimed convergence results imply the alternating scheme is a trustworthy solver for the penalty problem: the objective decreases monotonically and every limit point is stationary."],"supporting_citations":[{"why":"Defines sparse PCA, the base model that the paper extends.","marker":"[13]"},{"why":"SPCAFS, a single-sparsity $\\ell_{2,p}$ competitor whose method and results are extended and compared.","marker":"[16]"},{"why":"SPCA-PSD, the convex PSD reformulation used as a comparison baseline.","marker":"[17]"},{"why":"FSPCA, the $\\ell_{2,0}$-constrained PCA model that DSCOFS augments with an $\\ell_0$ constraint.","marker":"[21]"},{"why":"The exact penalty function method whose equivalence theorem justifies Algorithm 2 and the stationarity transfer.","marker":"[35]"},{"why":"Supplies the merit function and penalty term used in the X-subproblem reformulation.","marker":"[36]"},{"why":"Barzilai-Borwein step size used in the approximate gradient descent for the X-subproblem.","marker":"[37]"},{"why":"Hard thresholding operators that give the closed-form Y- and Z-updates.","marker":"[38]"},{"why":"The KL-property convergence framework used to prove global convergence of the alternating sequence.","marker":"[39]"}],"fun_headline_variants":["Double sparsity in PCA lifts feature selection accuracy by 3%+","Two sparsity types beat one: unsupervised feature selection gets sharper","Complementary sparsity constraints enhance unsupervised feature selection","PCA with dual sparsity: better features, proven convergence"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The convergence guarantee assumes the penalty objective is coercive—that $f(X,Y,Z)$ grows to infinity when the variables grow—and that each $X$-subproblem is solved by an exact global minimizer; the empirical gains also assume it is legitimate to tune the number of selected features and sparsity parameters on the true cluster labels.","fun_headline_variants_meta":{"raw":{"variants":["Double sparsity in PCA lifts feature selection accuracy by 3%+","Two sparsity types beat one: unsupervised feature selection gets sharper","Complementary sparsity constraints enhance unsupervised feature selection","PCA with dual sparsity: better features, proven convergence"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000598,"raw_usage":{"total_tokens":2842,"prompt_tokens":1034,"completion_tokens":1808,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":650,"completion_tokens_details":{"reasoning_tokens":1739}},"tokens_in":650,"tokens_out":1808,"duration_ms":12692,"temperature":1.0,"reasoning_tokens":1739,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:44:52.828785+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a data matrix $A$ with nonzero covariance and run Algorithm 1 from $X_0=Y_0=Z_0=tI$; along this ray $f = -t^2\\operatorname{Tr}(AA^{\\top})$, which goes to $-\\infty$ as $t$ grows, so the boundedness proof that invokes coercivity cannot hold. One can also check whether the $X$-updates returned by Algorithm 2 satisfy $X^{\\top}X \\approx I_m$ to the claimed tolerance; if they do not, the exact-minimizer premise of the descent inequality is violated.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines sparse PCA, the base model that the paper extends."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"SPCAFS, a single-sparsity $\\ell_{2,p}$ competitor whose method and results are extended and compared."},{"cited_title":"Fast Sparse PCA via Positive Semidefinite Projection for Unsupervised Feature Selection","cited_arxiv_id":"2309.06202","evidence_quote":"SPCA-PSD, the convex PSD reformulation used as a comparison baseline."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"FSPCA, the $\\ell_{2,0}$-constrained PCA model that DSCOFS augments with an $\\ell_0$ constraint."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The exact penalty function method whose equivalence theorem justifies Algorithm 2 and the stationarity transfer."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the merit function and penalty term used in the X-subproblem reformulation."},{"cited_title":"Huang, Y","cited_arxiv_id":null,"evidence_quote":"Barzilai-Borwein step size used in the approximate gradient descent for the X-subproblem."},{"cited_title":"Blumensath, M","cited_arxiv_id":null,"evidence_quote":"Hard thresholding operators that give the closed-form Y- and Z-updates."}],"review_version":1}