RECORD
PCA by Optimisation of Symmetric Functions has no Spurious Local Optima
Optimization and Control · Submitted 18 May 2018
abstract compiled · 2026-09-23 · no language-model review
The authors' main claim This paper introduces many other ways of performing PCA, with various geometric interpretations, and proves that the corresponding family of non-convex programs have no spurious local optima, while possessing only strict saddle points.
The compiled record
The paper's own sentences, arranged. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. Numbers in the tables are the authors', checked by code. No sentence in this record was written by a model.
Open the compiled record
The abstract, sentence by sentence
Principal Component Analysis (PCA) finds the best linear representation of data, and is an indispensable tool in many learning and inference tasks. Classically, principal components of a dataset are interpreted as the directions that preserve most of its "energy", an interpretation that is theoretically underpinned by the celebrated Eckart-Young-Mirsky Theorem. This paper introduces many other ways of performing PCA, with various geometric interpretations, and proves that the corresponding family of non-convex programs have no spurious local optima, while possessing only strict saddle points. These programs therefore loosely behave like convex problems and can be efficiently solved to global optimality, for example, with certain variants of the stochastic gradient descent. Beyond providing new geometric interpretations and enhancing our theoretical understanding of PCA, our findings might pave the way for entirely new approaches to structured dimensionality reduction, such as sparse PCA and nonnegative matrix factorisation. More specifically, we study an unconstrained formulation of PCA using determinant optimisation that might provide an elegant alternative to the deflating scheme commonly used in sparse PCA.
background contribution method result interpretation main claim
| # | sentence | role | the authors' own work | states the main conclusion | strength |
|---|---|---|---|---|---|
| s0 | Principal Component Analysis (PCA) finds the best linear representation of data, and is an indispensable tool in many learning and inference tasks. | background 1.00 | 0.05 | 0.02 | asserted 1.00 |
| s1 | Classically, principal components of a dataset are interpreted as the directions that preserve most of its "energy", an interpretation that is theoretically underpinned by the celebrated Eckart-Young-Mirsky Theorem. | background 1.00 | 0.04 | 0.02 | asserted 1.00 |
| s2 | This paper introduces many other ways of performing PCA, with various geometric interpretations, and proves that the corresponding family of non-convex programs have no spurious local optima, while possessing only strict saddle points. | contribution 0.76 | 0.96 | 0.91 | asserted 1.00 |
| s3 | These programs therefore loosely behave like convex problems and can be efficiently solved to global optimality, for example, with certain variants of the stochastic gradient descent. | interpretation 0.84 | 0.70 | 0.17 | asserted 0.77 |
| s4 | Beyond providing new geometric interpretations and enhancing our theoretical understanding of PCA, our findings might pave the way for entirely new approaches to structured dimensionality reduction, such as sparse PCA and nonnegative matrix factorisation. | future work 0.51 | 0.66 | 0.09 | speculative 0.73 |
| s5 | More specifically, we study an unconstrained formulation of PCA using determinant optimisation that might provide an elegant alternative to the deflating scheme commonly used in sparse PCA. | contribution 0.74 | 0.89 | 0.04 | speculative 0.84 |
What was examined, and what was not
- arXiv listing abstract v3
- full text
- figures and tables
- references and citing sentences
- relations to other papers
- P abstract
- S sections
- T body sentences
- C citations
- R cross-paper relations
The paper
Original abstract
Principal Component Analysis (PCA) finds the best linear representation of data, and is an indispensable tool in many learning and inference tasks. Classically, principal components of a dataset are interpreted as the directions that preserve most of its "energy", an interpretation that is theoretically underpinned by the celebrated Eckart-Young-Mirsky Theorem. This paper introduces many other ways of performing PCA, with various geometric interpretations, and proves that the corresponding family of non-convex programs have no spurious local optima, while possessing only strict saddle points. These programs therefore loosely behave like convex problems and can be efficiently solved to global optimality, for example, with certain variants of the stochastic gradient descent. Beyond providing new geometric interpretations and enhancing our theoretical understanding of PCA, our findings might pave the way for entirely new approaches to structured dimensionality reduction, such as sparse PCA and nonnegative matrix factorisation. More specifically, we study an unconstrained formulation of PCA using determinant optimisation that might provide an elegant alternative to the deflating scheme commonly used in sparse PCA.
Human review
How the literature uses it
2 arXiv papers cite this in 3 sentences
Papers citing it · 1 sentence without a confident label
Cites it as background 2 papers
-
We next recall a variant of the celebrated EYM theorem [ref] , which specifies the landscape of the PCA problem [ref].
Training Linear Neural Networks: Non-Local Convergence and Complexity Results · Landscape of Linear Networks · 0.80
-
Linear models are pervasive in data and computational sciences and, in particular, Principal Component Analysis (PCA) is an indispensable tool for detecting linear structure in collected data [ref] .
MOSES: A Streaming Algorithm for Linear Dimensionality Reduction · Introduction · 0.99
What it cites · 1 sentence without a confident label
Cites as background 5 papers
-
We remark that Theorem [ref] is in line with a recent trend in computational sciences to understand the geometry and performance of non-convex programs and algorithms [ref] .
When Are Nonconvex Problems Not Scary? · PCA by Determinant Optimisation · 0.93
-
We remark that Theorem [ref] is in line with a recent trend in computational sciences to understand the geometry and performance of non-convex programs and algorithms [ref] .
The Non-convex Geometry of Low-rank Matrix Optimization · PCA by Determinant Optimisation · 0.93
-
We remark that Theorem [ref] is in line with a recent trend in computational sciences to understand the geometry and performance of non-convex programs and algorithms [ref] .
Streaming Principal Component Analysis From Incomplete Data · PCA by Determinant Optimisation · 0.93
Discussion
Know something about this paper? Leave a comment, or highlight any passage above to comment on it.