pith. sign in

arxiv: 1906.11043 · v1 · pith:W7SQXMEEnew · submitted 2019-06-26 · 🧮 math.ST · stat.ME· stat.ML· stat.TH

Principal Component Analysis for Multivariate Extremes

Holger Drees , Anne Sabourin (LTCI) This is my paper

Pith reviewed 2026-05-25 15:04 UTC · model grok-4.3

classification 🧮 math.ST stat.MEstat.MLstat.TH
keywords multivariate extremesprincipal component analysisregular variationdimension reductionempirical risk minimizationheavy tailssubspace estimationreconstruction error
0
0 comments X

The pith

PCA on rescaled exceedances recovers the optimal low-dimensional subspace for multivariate extremes.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that principal component analysis applied to a rescaled version of radially thresholded observations from heavy-tailed vectors can identify the lower-dimensional subspace on which the limiting measure concentrates. Within an empirical risk minimization framework, it proves that the empirical squared reconstruction error converges uniformly to the true risk over all possible projection subspaces. As a direct consequence, the subspace minimizing the empirical risk converges in probability to the optimal one, measured by Hausdorff distance on the unit sphere. This dimension reduction is shown to come with finite-sample guarantees when observations are further rescaled to the unit ball. The approach matters for enabling more refined analysis of joint tail behavior in high dimensions by focusing on the relevant linear combinations.

Core claim

The first order behavior of multivariate heavy-tailed random vectors is ruled by a limit measure whose support is concentrated on a lower dimensional subspace. Applying PCA to rescaled radially thresholded observations, the squared reconstruction error's empirical risk converges uniformly to the true risk, implying that the best projection subspace converges in probability to the optimal one in Hausdorff distance between their intersections with the unit sphere. In addition, if the exceedances are re-scaled to the unit ball, finite sample uniform guarantees to the reconstruction error are obtained.

What carries the argument

Empirical risk minimization of the squared reconstruction error for PCA projections applied to rescaled exceedances over large radial thresholds.

If this is right

  • The estimated best projection subspace converges in probability to the true optimal subspace.
  • Finite sample uniform guarantees hold for the reconstruction error when exceedances are rescaled to the unit ball.
  • The method reduces dimension for refined statistical analysis of extremes.
  • Numerical experiments confirm relevance for practical data analysis.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The framework could be extended to other dimension reduction methods beyond PCA for extreme value analysis.
  • In applications like finance or climate modeling, this could improve prediction of joint extreme events by focusing on key directions.
  • One could test the method on simulated data with known subspaces to verify the convergence rates.
  • It suggests potential for combining with sparse methods to handle even higher dimensions.

Load-bearing premise

The support of the limiting measure is concentrated on a lower-dimensional linear subspace.

What would settle it

A simulation where the vector satisfies regular variation but the limit measure support fills the full space, checking whether the estimated subspace still converges in Hausdorff distance to a specific low-dimensional object.

Figures

Figures reproduced from arXiv: 1906.11043 by Anne Sabourin (LTCI), Holger Drees.

Figure 1
Figure 1. Figure 1: Mean empirical risk (left) and empirical risk for one sample (right) versus [PITH_FULL_IMAGE:figures/full_fig_p027_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Mean operator norm of the difference between the projection onto the true [PITH_FULL_IMAGE:figures/full_fig_p028_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: RMSE of the estimators of the probabilities (i)–(iv) based on [PITH_FULL_IMAGE:figures/full_fig_p028_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Mean empirical risk for PCA projecting onto a subspace of dimension 1 [PITH_FULL_IMAGE:figures/full_fig_p029_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: RMSE of the estimators of the probabilities (i)–(iv) based on [PITH_FULL_IMAGE:figures/full_fig_p030_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Mean empirical risk for PCA projecting onto a subspace of dimension 1 [PITH_FULL_IMAGE:figures/full_fig_p031_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: RMSE of the estimators of the probabilities (i)–(iv) based on [PITH_FULL_IMAGE:figures/full_fig_p032_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Mean empirical risk for PCA projecting onto a subspace of dimension 1 [PITH_FULL_IMAGE:figures/full_fig_p033_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: RMSE of the estimators of the probabilities (i)–(iv) based on [PITH_FULL_IMAGE:figures/full_fig_p034_9.png] view at source ↗
read the original abstract

The first order behavior of multivariate heavy-tailed random vectors above large radial thresholds is ruled by a limit measure in a regular variation framework. For a high dimensional vector, a reasonable assumption is that the support of this measure is concentrated on a lower dimensional subspace, meaning that certain linear combinations of the components are much likelier to be large than others. Identifying this subspace and thus reducing the dimension will facilitate a refined statistical analysis. In this work we apply Principal Component Analysis (PCA) to a re-scaled version of radially thresholded observations. Within the statistical learning framework of empirical risk minimization, our main focus is to analyze the squared reconstruction error for the exceedances over large radial thresholds. We prove that the empirical risk converges to the true risk, uniformly over all projection subspaces. As a consequence, the best projection subspace is shown to converge in probability to the optimal one, in terms of the Hausdorff distance between their intersections with the unit sphere. In addition, if the exceedances are re-scaled to the unit ball, we obtain finite sample uniform guarantees to the reconstruction error pertaining to the estimated projection sub-space. Numerical experiments illustrate the relevance of the proposed framework for practical purposes.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 3 minor

Summary. The manuscript develops a PCA procedure for identifying the lower-dimensional linear subspace supporting the limiting measure of a multivariate regularly varying random vector. Exceedances over a large radial threshold are rescaled and the problem is cast as empirical risk minimization of the squared reconstruction error. The central claims are that the empirical risk converges uniformly to the population risk over the Grassmannian of subspaces, that the argmin subspace converges in probability to the optimal subspace in Hausdorff distance on the unit sphere, and that finite-sample uniform bounds on the reconstruction error hold when the exceedances are further normalized to the unit ball. Numerical experiments are presented to illustrate practical performance.

Significance. If the stated uniform convergence and consistency results hold, the work supplies a theoretically grounded dimension-reduction tool for high-dimensional extreme-value analysis, allowing subsequent tail modeling to focus on the relevant linear combinations. The derivation via compactness of the Grassmannian, continuity of the risk map, and weak convergence of the rescaled point process is a clear technical strength; the paper thereby ships explicit consistency statements under standard regular-variation hypotheses without additional moment or entropy assumptions.

minor comments (3)
  1. [§2.2, §3] §2.2 and §3: the precise definition of the radial threshold sequence and its dependence on sample size should be stated explicitly when the uniform convergence is invoked, even if the argument is asymptotically insensitive to the choice.
  2. [Numerical experiments] The numerical section would benefit from reporting the dimension, sample size, and quantitative error metrics (e.g., Hausdorff distance or subspace angle) rather than qualitative illustrations alone.
  3. [§2] Notation for the Grassmannian manifold and the induced Hausdorff metric on the unit sphere should be introduced with a short self-contained paragraph in §2 to aid readers outside manifold optimization.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the detailed summary of our manuscript and for the positive assessment of its contributions. The recommendation of minor revision is noted. No specific major comments appear in the report, so we provide no point-by-point responses.

Circularity Check

0 steps flagged

No significant circularity

full rationale

The derivation establishes uniform convergence of the empirical risk to the population risk over projection subspaces via compactness of the Grassmannian, continuity of the reconstruction map, and weak convergence of the rescaled point process under regular variation; the Hausdorff consistency of the argmin then follows by standard arguments. No step reduces by definition to a fitted parameter, self-citation chain, or renamed input; the target subspace is defined independently via the limiting measure, and the results are externally falsifiable asymptotic statements.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

Central claims rest on the regular-variation assumption standard in extreme-value theory and on the modeling premise that the angular measure concentrates on a lower-dimensional subspace; no free parameters or invented entities are introduced in the abstract.

axioms (2)
  • domain assumption The random vector is multivariate regularly varying, so that exceedances over large radial thresholds converge to a non-degenerate limit measure.
    Invoked in the first sentence of the abstract as the framework governing first-order tail behavior.
  • domain assumption The support of the limit measure is concentrated on a lower-dimensional linear subspace.
    Stated as a reasonable assumption that motivates the dimension-reduction goal.

pith-pipeline@v0.9.0 · 5741 in / 1347 out tokens · 26296 ms · 2026-05-25T15:04:50.869676+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 34 canonical work pages · 2 internal anchors

  1. [1]

    Anderson, T. W. (1963). Asymptotic theory for principal component analysis. The Annals of Mathematical Statistics , 34(1):122--148

    work page 1963

  2. [2]

    Beirlant, J., Goegebeur, Y., Segers, J., and Teugels, J. L. (2006). Statistics of extremes: theory and applications . John Wiley & Sons

    work page 2006

  3. [3]

    Blanchard, G., Bousquet, O., and Zwald, L. (2007). Statistical properties of kernel principal component analysis. Machine Learning , 66(2-3):259--294

    work page 2007

  4. [4]

    Chautru, E. (2015). Dimension reduction in multivariate extreme value analysis. Electronic journal of statistics , 9(1):383--418

    work page 2015

  5. [5]

    and Sabourin, A

    Chiapino, M. and Sabourin, A. (2016). Feature clustering for extreme events analysis, with application to extreme stream-flow data. In International Workshop on New Frontiers in Mining Complex Patterns , pages 132--147. Springer

    work page 2016

  6. [6]

    Chiapino, M., Sabourin, A., and Segers, J. (2019). Identifying groups of variables with the potential of being large simultaneously. Extremes , 22(2):193--222

    work page 2019

  7. [7]

    Decompositions of Dependence for High-Dimensional Extremes

    Cooley, D. and Thibaud, E. (2016). Decompositions of dependence for high-dimensional extremes. arXiv preprint arXiv:1612.07190

    work page internal anchor Pith review Pith/arXiv arXiv 2016

  8. [8]

    H., de Haan, L., and Piterbarg, V

    Einmahl, J. H., de Haan, L., and Piterbarg, V. I. (2001). Nonparametric estimation of the spectral measure of an extreme value distribution. The Annals of Statistics , 29(5):1401--1423

    work page 2001

  9. [9]

    and Hitz, A

    Engelke, S. and Hitz, A. S. (2018). Graphical models for extremes. arXiv preprint arXiv:1812.01734

    work page arXiv 2018

  10. [10]

    and de Haan, L

    Ferreira, A. and de Haan, L. (2014). The generalized P areto process; with a view towards application and simulation. Bernoulli , 20(4):1717--1737

    work page 2014

  11. [11]

    Foug \`e res, A.-L., de Haan, L., and Mercadier, C. (2015). Bias correction in multivariate extremes. The Annals of Statistics , 43(2):903--934

    work page 2015

  12. [12]

    Gardes, L. (2018). Tail dimension reduction for extreme quantile estimation. Extremes , 21(1):57--95

    work page 2018

  13. [13]

    and Segers, J

    Genest, C. and Segers, J. (2009). Rank-based inference for bivariate extreme-value copulas. The Annals of Statistics , 37(5B):2990--3022

    work page 2009

  14. [14]

    Goix, N., Sabourin, A., and Cl \'e men c on, S. (2016). Sparse representation of multivariate extremes with applications to anomaly ranking. In AISTATS , pages 75--83

    work page 2016

  15. [15]

    Goix, N., Sabourin, A., and Cl \'e men c on, S. (2017). Sparse representation of multivariate extremes with applications to anomaly detection. Journal of Multivariate Analysis , 161:12--31

    work page 2017

  16. [16]

    Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association , 58:13--30

    work page 1963

  17. [17]

    and Lindskog, F

    Hult, H. and Lindskog, F. (2006). Regular variation for measures on metric spaces. Publ. Inst. Math.(Beograd)(NS) , 80(94):121--140

    work page 2006

  18. [18]

    $k$-means clustering of extremes

    Jan en, A. and Wan, P. (2019). k -means clustering of extremes. arXiv preprint arXiv:1904.02970

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  19. [19]

    and Gin \'e , E

    Koltchinskii, V. and Gin \'e , E. (2000). Random matrix approximation of spectra of integral operators. Bernoulli , 6(1):113--167

    work page 2000

  20. [20]

    and Lounici, K

    Koltchinskii, V. and Lounici, K. (2017). New asymptotic results in principal component analysis. Sankhya A , 79(2):254--297

    work page 2017

  21. [21]

    I., and Roy, J

    Lindskog, F., Resnick, S. I., and Roy, J. (2014). Regularly varying measures on metric spaces: Hidden regular variation and hidden jumps. Probability Surveys , 11:270--314

    work page 2014

  22. [22]

    McDiarmid, C. (1998). Concentration. In Probabilistic methods for algorithmic discrete mathematics , pages 195--248. Springer

    work page 1998

  23. [23]

    A., Ribatet, M., and Sisson, S

    Padoan, S. A., Ribatet, M., and Sisson, S. A. (2010). Likelihood-based inference for max-stable processes. Journal of the American Statistical Association , 105(489):263--277

    work page 2010

  24. [24]

    Resnick, S. I. (2007). Heavy-tail phenomena: probabilistic and statistical modeling . Springer Science & Business Media

    work page 2007

  25. [25]

    Resnick, S. I. (2013). Extreme values, regular variation and point processes . Springer

    work page 2013

  26. [26]

    Rootz \'e n, H., Segers, J., and Wadsworth, J. L. (2018). Multivariate generalized P areto distributions: Parametrizations, representations, and properties. Journal of Multivariate Analysis , 165:117--131

    work page 2018

  27. [27]

    Schlather, M. (2002). Models for stationary max-stable random fields. Extremes , 5(1):33--44

    work page 2002

  28. [28]

    Seber, G. A. F. (1984). Multivariate Observations . Wiley

    work page 1984

  29. [29]

    Segers, J. (2012). Max-stable models for multivariate extremes. REVSTAT , 10:61--82

    work page 2012

  30. [30]

    K., Cristianini, N., and Kandola, J

    Shawe-Taylor, J., Williams, C. K., Cristianini, N., and Kandola, J. (2005). On the eigenspectrum of the gram matrix and the generalization error of kernel-pca. Information Theory, IEEE Transactions on , 51(7):2510--2522

    work page 2005

  31. [31]

    S., Wadsworth, J

    Simpson, E. S., Wadsworth, J. L., and Tawn, J. A. (2018). Determining the dependence structure of multivariate extremes. arXiv preprint arXiv:1809.01606

    work page arXiv 2018

  32. [32]

    Stephenson, A. (2003). Simulating multivariate extreme value distributions of logistic type. Extremes , 6:49--59

    work page 2003

  33. [33]

    Van der Vaart, A. W. (2000). Asymptotic statistics . Cambridge University Press

    work page 2000

  34. [34]

    and Blanchard, G

    Zwald, L. and Blanchard, G. (2006). On the convergence of eigenspaces in kernel principal component analysis. In Advances in neural information processing systems , pages 1649--1656

    work page 2006