Pith. sign in

REVIEW 3 major objections 7 minor 33 references

msPCA: An R Package for Sparse PCA with Multiple Components

T0 review · 3 major / 7 minor · reviewed 2026-07-07 · glm-5.2

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read R package enforces non-redundant sparse PCA at scale

desk verdict Solid software paper with one real gap in experimental design read the letter →

arxiv 2607.05229 v1 pith:NDTL7XVJ submitted 2026-07-06 stat.ML cs.LGstat.ME

Ryan Cory-Wright , Jean Pauphilet This is my paper
classification stat.MLcs.LGstat.ME
keywords sparsePCAmultiplecomponentsnon-redundancyconstraintsorthogonalityzeropairwisecorrelationLagrangianrelaxationalternatingmaximizationRpackage
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper presents msPCA, an R package for sparse principal component analysis with multiple components. The central claim is that among comparable R packages, msPCA is the only one supporting both orthogonality and zero-pairwise-correlation constraints on multi-component sparse PCA, while giving the user an explicit tolerance parameter to control how far the returned solution may violate those constraints. The mechanism is a Lagrangian alternating maximization algorithm: coupling constraints between components are replaced by quadratic penalties, which decomposes the problem into independent single-component sparse PCA subproblems each solved by the Truncated Power Method. Penalty parameters start at zero and increase across iterations, progressively driving constraint violations toward the user-specified tolerance. Benchmarks on four real datasets show msPCA consistently returns solutions with feasibility violations at or below its default tolerance of 10^{-4}, while achieving variance-explained performance competitive with the best alternatives and scaling to thousands of features when operating directly on the data matrix rather than the covariance matrix.

What carries the argument

Lagrangian alternating maximization with progressive penalty increase; Truncated Power Method for single-component sparse PCA subproblems; implicit matrix-vector representation avoiding O(p^2) covariance materialization when operating on raw data.

What would settle it

If msPCA were run on a dataset where the penalty schedule fails to drive constraint violations below the user tolerance within the iteration budget, it would return an infeasible solution, contradicting the claim of controlled feasibility.

Watch

Extended reading notes

Core claim

The paper's central contribution is the demonstration that explicitly penalizing non-redundancy constraints in multi-component sparse PCA, rather than relying on deflation or projection heuristics, yields solutions with controlled and predictable feasibility violations across a range of datasets and sparsity levels. The Lagrangian scheme decomposes the coupled multi-component problem into a sequence of single-component sparse eigenvector problems against a perturbed covariance matrix, where the perturbation is induced by penalty terms on the constraint violations. This approach is the only one among seven compared packages that supports both orthogonality (u_t^T u_t' = 0) and zero pairwise (

Load-bearing premise

The penalty parameters are increased according to a hand-tuned heuristic schedule (exploration phase, ratio-based updates, and a boost in the final iterations) with no formal guarantee that the algorithm will find a feasible solution within the iteration budget, especially since the Truncated Power Method solves subproblems approximately rather than exactly.

Editorial extensions

If this is right

  • Practitioners who need statistically uncorrelated sparse components for downstream regression or classification can now obtain them with explicit feasibility guarantees, rather than accepting whatever correlation structure a deflation heuristic happens to produce.
  • The two constraint types (orthogonality vs. zero correlation) can yield meaningfully different factor compositions on correlated data, as shown in the S&P 500 case study, so the choice is not merely cosmetic and should be reported in applied work.
  • The type='X' interface that avoids forming the p x p covariance matrix makes sparse PCA practical in high-dimensional, low-sample-size regimes (p >> n) where the covariance matrix is rank-deficient or expensive to store.
  • Existing sparse PCA packages that rely on deflation or projection may silently return non-orthogonal or correlated components, which could compromise downstream analyses that assume non-redundancy.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the penalty update schedule were replaced by a principled adaptive scheme with formal convergence guarantees (e.g., augmented Lagrangian methods with dual updates), the feasibility guarantee could be made rigorous even with inexact subproblem solves, addressing the paper's acknowledged limitation.
  • The implicit matrix-vector representation could be extended to streaming or distributed settings, since X^T(Xβ) decomposes naturally across data partitions, potentially enabling sparse PCA on datasets too large to fit in memory.
  • The distinction between orthogonality and zero-correlation constraints suggests a broader family of non-redundancy definitions (e.g., zero partial correlation, or constraints on specific cross-loadings) that could be incorporated into the same Lagrangian framework with different choices of the constraint matrix C.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 7 minor

Summary. This manuscript presents msPCA, an open-source R package for sparse PCA with multiple components. The package implements a Lagrangian alternating maximization algorithm that penalizes coupling constraints (orthogonality or zero pairwise correlation) and progressively increases penalties to drive feasibility violations toward a user-specified tolerance. Subproblems are solved via the Truncated Power Method (TPM). The paper compares msPCA against seven competing R/Python packages on four benchmark datasets, reporting fraction of variance explained (FVE), orthogonality violation, and runtime. A case study on S&P 500 returns illustrates the practical difference between orthogonality and zero-correlation constraints. The package is available on CRAN with replication materials.

Significance. The software contribution is clear and useful: msPCA is the only R package among those compared that supports both orthogonality and zero-pairwise-correlation non-redundancy constraints with an explicit feasibility tolerance parameter. The implicit mat-vec representation avoiding O(p^2) covariance matrix formation (type = 'X') is a practical scaling improvement, demonstrated on the riboflavin dataset (p=4,088, n=71) where runtime drops from ~72s to ~2s. The benchmarks span four datasets and seven competing methods across multiple metrics. The package ships on CRAN with replication scripts and a vignette, which is appropriate for a software paper.

major comments (3)
  1. Section 6.1 states: 'For each method and each dataset, metrics are computed on a single run (with all seeds fixed and provided in the replication scripts).' This is in direct tension with the central claim that msPCA 'consistently returns solutions with feasibility violations at or below the user-specified tolerance across all benchmark datasets.' Section 7 acknowledges that 'a small fraction of runs may fail to find a feasible solution.' A single run per configuration cannot establish consistency or assess the practical magnitude of the failure rate. The authors should either (a) soften the 'consistently' language to match what a single run can support, or (b) run multiple seeds (e.g., 20-50) and report the fraction of runs achieving feasibility within tolerance. This is load-bearing because controlled feasibility is the package's distinguishing feature (Table 1).
  2. Section 5.3 reports that at k=5 with orthogonality constraints on the S&P 500 data, the violation is 2.2×10^-4, which exceeds the default tolerance of 10^-4. The paper notes this is 'marginally above' the tolerance, but this directly contradicts the claim in Section 6.3 that msPCA 'consistently returns small orthogonality violations on all four datasets, around 10^-4 (the default feasibility tolerance) or below.' The authors should reconcile this: either the claim should be qualified (e.g., 'at or near the tolerance') or the algorithm should be examined for why it returns solutions above the specified tolerance.
  3. The FVE comparisons in Tables 2-5 lack any variance estimates or standard errors. Given that TPM uses random restarts (Algorithm 2) and the penalty update is heuristic, it is unclear whether the reported FVE differences are robust or seed-dependent. For example, on mtcars (Table 2), msPCA reports FVE=0.829 vs. nsprcomp's 0.832 — a negligible difference that could easily be noise. The authors should either report FVE over multiple seeds with standard deviations, or explicitly acknowledge that single-run FVE comparisons are illustrative rather than definitive.
minor comments (7)
  1. Section 3.2: The penalty update schedule (15% exploration phase, ratio-based update, 5× boost in last 25% of iterations) is described qualitatively but the exact update rule is not given. While the authors cite Cory-Wright and Pauphilet (2026) for details, a brief formal statement of the update rule would make the paper more self-contained.
  2. Table 1: The entry for nsprcomp lists 'penalty parameter γ' under 'Control parameter,' but the text in Section 6.1 clarifies that nsprcomp::nscumcomp() has a default γ=0 (i.e., no penalty). This could be misleading; consider adding a footnote clarifying that the default disables the penalty.
  3. Section 6.1: The Pitprops dataset pseudo-data generation (MASS::mvrnorm with n=500) is mentioned in passing. It would help to state the seed used for reproducibility, or confirm it is in the replication scripts.
  4. Tables 2-5: The sparsity columns show actual nonzero counts per PC, which differ from the target k for several methods. This is clearly noted in the discussion, but a brief footnote on each table reminding the reader that entries reflect actual (not target) sparsity would improve readability.
  5. Section 5.1: The data URL footnote references a Kaggle dataset. Consider confirming the license terms are compatible with the paper's CC-BY 4.0 license, or noting any restrictions.
  6. Algorithm 1: The feasibility check uses the sum of norm violations and constraint violations compared to η. It would help to clarify whether η bounds the sum or the maximum individual violation, as this affects interpretation of the reported violation values.
  7. The abstract states 'controlled feasibility violations' which is appropriately hedged, but the phrase 'consistently' in Section 6.3 is stronger. Aligning the language across abstract, body, and conclusion would strengthen the paper's credibility.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive report. All three major comments are well-taken and concern the same underlying issue: single-run benchmark results cannot support the consistency language used in the manuscript. We will (a) run multi-seed experiments (30 seeds per configuration) and report FVE means/standard deviations and feasibility-achievement rates, (b) soften all 'consistently' claims to match what the evidence supports, and (c) reconcile the Section 5.3 tolerance-exceedance with the Section 6.3 claim. All changes will be incorporated in the revision.

read point-by-point responses

  1. Referee: Section 6.1 states metrics are computed on a single run, which is in tension with the claim that msPCA 'consistently returns solutions with feasibility violations at or below the user-specified tolerance.' The referee requests either softening the language or running multiple seeds and reporting the fraction achieving feasibility.

    Authors: The referee is correct. A single run per configuration cannot establish consistency, and the tension between the single-run methodology and the 'consistently' language is real. We will address this by running 30 seeds per configuration across all benchmark datasets and reporting the fraction of runs achieving feasibility within tolerance, along with mean and standard deviation of the feasibility violation. We will also soften the 'consistently' language throughout to accurately reflect the empirical failure rate. We note that the algorithm's feasibility tracking mechanism (Algorithm 1 records the best feasible solution across all outer iterations) provides a structural reason to expect high feasibility-achievement rates, but agree this must be demonstrated empirically rather than asserted. revision: yes

  2. Referee: Section 5.3 reports a violation of 2.2e-4 at k=5 with orthogonality constraints on S&P 500 data, exceeding the default tolerance of 1e-4, which contradicts the Section 6.3 claim that msPCA 'consistently returns small orthogonality violations on all four datasets, around 10^-4 or below.'

    Authors: The referee has identified a genuine internal inconsistency. The 2.2e-4 violation at k=5 on the S&P 500 data does exceed the default tolerance of 1e-4, and the Section 6.3 claim should not have been stated without qualification. We will revise the language in Section 6.3 to say 'at or near the default feasibility tolerance' rather than 'at or below,' and will add an explicit footnote acknowledging the k=5 S&P 500 case. We will also add a brief algorithmic remark noting that when the penalty update scheme does not fully converge within the iteration budget, the returned solution may marginally exceed the tolerance, which is why the algorithm returns the best feasible solution found across all iterations rather than guaranteeing the final iterate is feasible. revision: yes

  3. Referee: The FVE comparisons in Tables 2-5 lack variance estimates or standard errors. Given TPM's random restarts and the heuristic penalty update, it is unclear whether reported FVE differences are robust or seed-dependent. The mtcars example (FVE 0.829 vs 0.832) could easily be noise.

    Authors: The referee is right that single-run FVE comparisons cannot distinguish genuine performance differences from seed-dependent variation, particularly for small differences like the 0.829 vs 0.832 gap on mtcars. We will report FVE mean and standard deviation across 30 seeds for msPCA on all benchmark datasets. For competing methods, we will also run multiple seeds where the underlying algorithm is stochastic (elasticnet, sparsepca, amanpg, scikit-learn) and note which methods are deterministic. We will add an explicit statement that single-run FVE comparisons should be interpreted as illustrative rather than definitive, and will flag the mtcars comparison specifically as within noise. We note that for the orthogonality violation metric, the multi-seed results are more directly relevant to the package's distinguishing feature (controlled feasibility), so we will emphasize those results. revision: yes

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity found; self-citation to prior methodology work is not load-bearing for the software paper's empirical claims.

full rationale

This is a software package paper, not a derivation paper. The central claims are: (1) msPCA supports both orthogonality and zero-correlation constraints, (2) it returns solutions with feasibility violations at or below tolerance, and (3) it achieves competitive FVE on external benchmark datasets. These claims are substantiated by external benchmarks (mtcars, Pitprops, breast cancer, riboflavin, S&P 500) and comparison against seven competing packages, not by theoretical derivation. The paper does cite Cory-Wright and Pauphilet (2026) for the algorithm and theoretical feasibility guarantee, and the authors overlap. However, this citation is used to attribute the algorithm's origin and theoretical properties, not to define the paper's central results circularly. The paper explicitly acknowledges the heuristic nature of the penalty update scheme and that 'a small fraction of runs may fail to find a feasible solution,' which is an honest limitation rather than a circular construction. The feasibility tracking mechanism (returning the best feasible solution found, or least-infeasible if none found) is a straightforward algorithmic feature, not a definitional trick. The benchmark results are computed on external datasets against external competitors. No step in the paper's argument reduces to its own inputs by construction. The self-citation is standard attribution of prior methodology work, not a load-bearing circular dependency.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

No new entities are introduced. The algorithm operates on standard mathematical objects (covariance matrices, loading vectors, penalty parameters).

free parameters (5)
  • Penalty schedule thresholds (15%, 25%, 5× boost) = 0.15, 0.25, 5
    Hand-tuned heuristic schedule for penalty parameter updates in Section 3.2, not derived from theory.
  • maxIter = 200
    Default outer iteration limit, chosen empirically.
  • feasibilityTolerance (η) = 1e-04
    Default feasibility tolerance, user-settable but default chosen empirically.
  • maxRestartTPM / minRestartTPM = 30 / 20
    Random restart counts for Truncated Power Method, chosen empirically.
  • timeLimitTPM = 20
    Time limit for TPM subproblems in seconds, chosen empirically.
assumptions (3)
  • domain assumption The Truncated Power Method (Yuan and Zhang, 2013) finds high-quality solutions to single-component sparse PCA subproblems.
    Section 3.1: each subproblem is solved via TPM. The quality of the overall solution depends on TPM finding good sparse eigenvectors of the perturbed matrix.
  • domain assumption Progressively increasing penalty parameters drives constraint violations toward zero when subproblems are solved exactly.
    Section 3.1 and Algorithm 1: the penalty increase scheme is the core mechanism. Theoretical justification is cited from Cory-Wright and Pauphilet (2026, Section 3) but requires exact subproblem solutions, which TPM does not guarantee.
  • standard math The perturbed covariance matrix Σ̃_t can be made positive semidefinite by a diagonal shift λ₀I without changing the optimal solution.
    Section 3.1: justified because all feasible vectors have unit norm, so adding λ₀I shifts all eigenvalues equally and does not change the optimal eigenvector.

how reviews work

0 comments
Cite this review

Pith. "Pith review of msPCA: An R Package for Sparse PCA with Multiple Components." pith.science (2026). https://pith.science/paper/NDTL7XVJ

@misc{pith2026260705229,
  author       = {Pith},
  title        = {Pith review of: msPCA: An R Package for Sparse PCA with Multiple Components},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NDTL7XVJ}},
  note         = {Machine review of arXiv:2607.05229}
}
read the original abstract

We present msPCA: an open-source R package for sparse principal component analysis with multiple components. It implements an alternating maximization algorithm to generate a set of sparse loading vectors that collectively explain a large fraction of the variance in a dataset, while remaining non-redundant. The algorithm supports two definitions of non-redundancy: either orthogonality of the loading vectors or zero pairwise correlation between principal components (PCs). In the reported benchmarks, msPCA solves sparse PCA problems with thousands of features, achieving competitive runtimes while producing sparse components with controlled feasibility violations and a high fraction of variance explained.

Figures

Figures reproduced from arXiv: 2607.05229 by the authors.

Figure 1
Figure 1. reports FVE, orthogonality violation (violoff(I, U)), and uncorrelatedness violation (violoff(Σ, U)) as a function of k, for msPCA with each constraint type. We compare with the function nsprcomp::nsprcomp of the nsprcomp package (Sigg, 2019) at the same sparsity budgets. 0.025 0.050 0.075 0.100 10 20 30 Sparsity budget k FVE nsprcomp orthogonality zero−correlation 0.0 0.1 0.2 0.3 0.4 0.5 10 20 30 Sparsity budget k … view at source ↗
Figure 2
Figure 2. S&P 500 case study. Loadings of the 4 PCs (sparsity k = 10) returned by msPCA with orthogonality (left side) or zero-correlation (right side) constraints. 6 Benchmarking We compare msPCA against seven competing packages: elasticnet (Zou et al., 2006), PMA (Witten et al., 2009), sparsepca (Erichson et al., 2020), mixOmics (Rohart et al., 2017), and nsprcomp (Sigg, 2019) (all R); amanpg (Chen et al., 2020) (R, also av… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

33 extracted references · 33 canonical work pages

  1. [1]

    PLOS Computational Biology , volume=

    Rohart, Florian and Gautier, Beno. PLOS Computational Biology , volume=. 2017 , publisher=

    work page 2017

  2. [2]

    Sigg, Christian D. , year=

    work page

  3. [3]

    Journal of Statistical Software , volume=

    Rcpp: Seamless R and C++ integration , author=. Journal of Statistical Software , volume=

    work page

  4. [4]

    Cancer Cell , volume=

    Genomic and transcriptional aberrations linked to breast cancer pathophysiologies , author=. Cancer Cell , volume=. 2006 , publisher=

    work page 2006

  5. [5]

    Applied Statistics , volume=

    Two Case Studies in the Application of Principal Component Analysis , author=. Applied Statistics , volume=. 1967 , publisher=

    work page 1967

  6. [6]

    Biostatistics , volume=

    A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis , author=. Biostatistics , volume=. 2009 , publisher=

    work page 2009

  7. [7]

    SIAM Journal on Applied Mathematics , volume=

    Sparse principal component analysis via variable projection , author=. SIAM Journal on Applied Mathematics , volume=. 2020 , publisher=

    work page 2020

  8. [8]

    Journal of Computational and Graphical Statistics , volume=

    Sparse principal component analysis , author=. Journal of Computational and Graphical Statistics , volume=. 2006 , publisher=

    work page 2006

Show all 33 references
  1. [9]

    Journal of Machine Learning Research , volume=

    Sparse PCA via covariance thresholding , author=. Journal of Machine Learning Research , volume=

  2. [10]

    Mathematical Programming , volume=

    An augmented Lagrangian approach for sparse principal component analysis , author=. Mathematical Programming , volume=. 2012 , publisher=

    2012

  3. [11]

    Mathematical Programming Computation , volume=

    Certifiably optimal sparse principal component analysis , author=. Mathematical Programming Computation , volume=. 2019 , publisher=

    2019

  4. [12]

    Annals of Statistics (to appear) , year=

    Sparse PCA: A new scalable estimator based on integer programming , author=. Annals of Statistics (to appear) , year=

  5. [13]

    , author=

    Truncated Power Method for Sparse Eigenvalue Problems. , author=. Journal of Machine Learning Research , volume=

  6. [14]

    Journal of Machine Learning Research , volume=

    Solving large-scale sparse PCA to certifiable (near) optimality , author=. Journal of Machine Learning Research , volume=

  7. [15]

    Operations Research , url=

    Sparse PCA With Multiple Components , author=. Operations Research , url=

  8. [16]

    On lines and planes of closest fit to systems of points in space , author=

    LIII. On lines and planes of closest fit to systems of points in space , author=. The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science , volume=. 1901 , publisher=

    1901

  9. [17]

    A direct formulation for sparse

    d'Aspremont, Alexandre and El Ghaoui, Laurent and Jordan, Michael I and Lanckriet, Gert RG , journal=. A direct formulation for sparse. 2007 , publisher=

    2007

  10. [18]

    Journal of the American Statistical Association , volume=

    On consistency and sparsity for principal components analysis in high dimensions , author=. Journal of the American Statistical Association , volume=. 2009 , publisher=

    2009

  11. [19]

    Statistics Surveys , volume=

    Interpretable machine learning: Fundamental principles and 10 grand challenges , author=. Statistics Surveys , volume=

  12. [20]

    Annual Review of Statistics and Its Application , volume=

    High-dimensional statistics with a view toward applications in biology , author=. Annual Review of Statistics and Its Application , volume=. 2014 , publisher=

    2014

  13. [21]

    , author=

    Analysis of a complex of statistical variables into principal components. , author=. Journal of Educational Psychology , volume=. 1933 , publisher=

    1933

  14. [22]

    Proceedings of the 25th International Conference on Machine Learning , pages=

    Expectation-maximization for sparse and non-negative PCA , author=. Proceedings of the 25th International Conference on Machine Learning , pages=

  15. [23]

    Journal of Multivariate Analysis , volume=

    Sparse principal component analysis via regularized low rank matrix approximation , author=. Journal of Multivariate Analysis , volume=. 2008 , publisher=

    2008

  16. [24]

    Statistical Applications in Genetics and Molecular Biology , volume=

    Extensions of sparse canonical correlation analysis with applications to genomic data , author=. Statistical Applications in Genetics and Molecular Biology , volume=

  17. [25]

    Advances in Neural Information Processing Systems , volume=

    Deflation methods for sparse PCA , author=. Advances in Neural Information Processing Systems , volume=

  18. [26]

    2023 , howpublished =

    The. 2023 , howpublished =

    2023

  19. [27]

    , author=

    Generalized power method for sparse principal component analysis. , author=. Journal of Machine Learning Research , volume=

  20. [28]

    Fantope projection and selection: A near-optimal convex relaxation of sparse

    Vu, Vincent Q and Cho, Juhee and Lei, Jing and Rohe, Karl , journal=. Fantope projection and selection: A near-optimal convex relaxation of sparse

  21. [29]

    Orthogonal sparse

    Benidis, Konstantinos and Sun, Ying and Babu, Prabhu and Palomar, Daniel P , journal=. Orthogonal sparse. 2016 , publisher=

    2016

  22. [30]

    Journal of Statistical Software , volume=

    Fast and elegant numerical linear algebra using the RcppEigen package , author=. Journal of Statistical Software , volume=

  23. [31]

    Proximal gradient method for nonsmooth optimization over the

    Chen, Shixiang and Ma, Shiqian and So, Anthony Man-Cho and Zhang, Tong , journal=. Proximal gradient method for nonsmooth optimization over the. 2020 , publisher=

    2020

  24. [32]

    and Tang, Yuan , year=

    Ushey, Kevin and Allaire, J.J. and Tang, Yuan , year=

  25. [33]

    Scikit-learn: Machine learning in

    Pedregosa, Fabian and Varoquaux, Ga. Scikit-learn: Machine learning in. Journal of Machine Learning Research , volume=

Pith tools

Reviewed July 7, 2026 · model on record in the stance chip above.