pith. sign in

arxiv: 2408.15701 · v1 · submitted 2024-08-28 · 📊 stat.ME · stat.CO

Robust discriminant analysis

Mia Hubert , Jakob Raymaekers , Peter J. Rousseeuw This is my paper

Pith reviewed 2026-05-23 22:25 UTC · model grok-4.3

classification 📊 stat.ME stat.CO
keywords discriminant analysisrobust statisticsoutliersmislabeled datalocation and scatterclassificationgraphical diagnostics
0
0 comments X

The pith

Standard discriminant analysis fails under outliers or label errors, but robust versions using resistant location and scatter estimates stay reliable.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Discriminant analysis classifies observations by modeling the center and spread of each group. Classical versions rely on the sample mean and covariance, which shift dramatically when even a few points are outliers or carry wrong labels. The paper shows this sensitivity with examples and then surveys robust alternatives that substitute estimators designed to resist such contamination. These replacements produce decision rules whose performance holds up better on contaminated data. The review also includes diagnostic plots to spot the influential cases.

Core claim

The paper establishes that classical discriminant analysis uses non-robust estimators of location and scatter for each class, rendering it sensitive to outliers and mislabeled observations, and reviews a range of robust discriminant analysis methods that substitute resistant estimators of location and scatter to obtain classifications that remain reliable in the presence of deviating cases, together with graphical tools for visualizing results.

What carries the argument

Robust estimates of location and scatter computed separately for each class, which replace the arithmetic mean and sample covariance matrix when forming the discriminant rules.

If this is right

  • Classifications obtained from robust discriminant analysis stay stable when a moderate fraction of the data are outliers.
  • Mislabeled observations exert reduced influence on the estimated class boundaries.
  • Graphical diagnostic tools allow identification of the suspicious points that affect the analysis.
  • Both linear and quadratic forms of discriminant analysis can be made robust by the same replacement of estimators.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same substitution of robust estimators could be tested in related supervised methods such as logistic regression to check for similar gains.
  • In high-dimensional settings the robust estimators would need dimension-reduction steps first, an extension the paper does not address.
  • Benchmark comparisons on public datasets with known contamination levels would give quantitative evidence of the improvement over classical rules.
  • The approach suggests that any classification procedure relying on second-moment summaries could benefit from analogous robust replacements.

Load-bearing premise

Robust estimators of location and scatter can be computed for each class and will yield classifications that remain reliable when the data contain outliers or label errors.

What would settle it

A controlled experiment on data with 5-10 percent added outliers or swapped labels where the robust methods produce higher misclassification rates than classical discriminant analysis would refute the reliability claim.

Figures

Figures reproduced from arXiv: 2408.15701 by Jakob Raymaekers, Mia Hubert, Peter J. Rousseeuw.

Figure 1
Figure 1. Figure 1: The effect of label and measurement noise on CQDA. Top row: two uncontaminated [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: The effect of label and measurement noise on robust QDA. The solid curve in the [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Stacked plot of the data in Figure [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: RQDA silhouette plot of the bivariate data from Figure [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Quasi residual plot with the distance to the predicted class on the horizontal axis. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Class maps of the simulated data of Figure [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Stacked plots of the floral buds data where the outlier class is defined (left) through [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Silhouette plot of the floral bud data. To understand this difference, look at the plot of the squared robust distances of the buds versus quantiles of the χ 2 6 distribution in [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Quasi residual plots of the floral bud data, with the distance to the predicted class [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Class maps of the floral bud data [PITH_FULL_IMAGE:figures/full_fig_p017_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Chi-squared Q-Q plot of the squared robust distances of the bud class. The gray [PITH_FULL_IMAGE:figures/full_fig_p018_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Quasi residual plot of the floral bud data with the difference between variables 1 [PITH_FULL_IMAGE:figures/full_fig_p019_12.png] view at source ↗
read the original abstract

Discriminant analysis (DA) is one of the most popular methods for classification due to its conceptual simplicity, low computational cost, and often solid performance. In its standard form, DA uses the arithmetic mean and sample covariance matrix to estimate the center and scatter of each class. We discuss and illustrate how this makes standard DA very sensitive to suspicious data points, such as outliers and mislabeled cases. We then present an overview of techniques for robust DA, which are more reliable in the presence of deviating cases. In particular, we review DA based on robust estimates of location and scatter, along with graphical diagnostic tools for visualizing the results of DA.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

0 major / 3 minor

Summary. The manuscript reviews how classical discriminant analysis (DA), which relies on the arithmetic mean and sample covariance for each class, is highly sensitive to outliers and mislabeled observations. It then surveys robust DA approaches that substitute robust estimators of location and scatter, and discusses associated graphical diagnostic tools for visualizing results under contamination.

Significance. As a descriptive review rather than a source of new theorems or algorithms, the paper's value lies in consolidating known results from robust statistics literature and illustrating the practical limitations of standard DA. If the survey is balanced and cites the key references accurately, it could serve as a useful entry point for practitioners, but it does not advance novel methodology or provide original empirical validation.

minor comments (3)
  1. The abstract states that robust DA techniques 'are more reliable in the presence of deviating cases' but does not specify the contamination models or breakdown points considered; adding a brief qualification would improve precision without altering the review character.
  2. The description of standard DA sensitivity would benefit from a short numerical illustration (e.g., a small contaminated dataset) early in the text to make the claim concrete before moving to the overview of robust methods.
  3. Ensure that all cited robust estimators (e.g., MCD, M-estimators) are accompanied by at least one key reference in the first mention to allow readers to locate the original methodological papers.

Simulated Author's Rebuttal

0 responses · 0 unresolved

We thank the referee for the positive evaluation and recommendation of minor revision. The report provides no specific major comments to address point-by-point, and we agree with the characterization of the manuscript as a descriptive review consolidating existing results rather than introducing new methodology.

Circularity Check

0 steps flagged

Review paper with no derivation chain or predictions

full rationale

This manuscript is an overview article surveying standard discriminant analysis sensitivity to outliers and existing robust alternatives based on robust location/scatter estimators. No new theorems, derivations, fitted parameters, or predictions are advanced; the text reviews prior techniques and diagnostics from the robust statistics literature without introducing self-referential steps or reducing claims to inputs by construction. The central claims are descriptive restatements of established properties, with no load-bearing self-citations or ansatzes that could create circularity.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

This is a review paper; the abstract introduces no new free parameters, axioms, or invented entities.

pith-pipeline@v0.9.0 · 5626 in / 1006 out tokens · 23734 ms · 2026-05-23T22:25:31.085145+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

27 extracted references · 27 canonical work pages

  1. [1]

    \ Wilms, I

    Aerts2017 APACrefauthors Aerts, S. \ Wilms, I. APACrefauthors \ 2017 . Cellwise robust regularized discriminant analysis Cellwise robust regularized discriminant analysis . Statistical Analysis and Data Mining: The ASA Data Science Journal 10 6 436--447

    work page 2017

  2. [2]

    APACrefauthors \ 1978

    Campbell1978 APACrefauthors Campbell, N A. APACrefauthors \ 1978 . The Influence Function as an Aid in Outlier Detection in Discriminant Analysis The influence function as an aid in outlier detection in discriminant analysis . Applied Statistics 27 3 251--258

    work page 1978

  3. [3]

    \ Rousseeuw, P J

    Chork:DA-MVE APACrefauthors Chork, C. \ Rousseeuw, P J. APACrefauthors \ 1992 . Integrating a high-breakdown option into discriminant analysis in exploration geochemistry Integrating a high-breakdown option into discriminant analysis in exploration geochemistry . Journal of Geochemical Exploration 43 3 191--203

    work page 1992

  4. [4]

    \ Dehon, C

    Croux:Discrim APACrefauthors Croux, C. \ Dehon, C. APACrefauthors \ 2001 . Robust linear discriminant analysis using S -estimators Robust linear discriminant analysis using S -estimators . The Canadian Journal of Statistics 29 3 473--493

    work page 2001

  5. [5]

    , Hubert, M

    DeKetelaere:RTDetMCD APACrefauthors De Ketelaere, B. , Hubert, M. , Raymaekers, J. , Rousseeuw, P J. \ Vranckx, I. APACrefauthors \ 2020 . Real-time outlier detection for large datasets by RT-DetMCD Real-time outlier detection for large datasets by RT-DetMCD . Chemometrics and Intelligent Laboratory Systems 199 103957

    work page 2020

  6. [6]

    \ Fung, W

    He:Discrim APACrefauthors He, X. \ Fung, W. APACrefauthors \ 2000 . High breakdown estimation for multiple populations with applications to discriminant analysis High breakdown estimation for multiple populations with applications to discriminant analysis . Journal of Multivariate Analysis 72 151--162

    work page 2000

  7. [7]

    , Debruyne, M

    Hubert:WIRE-MCD2 APACrefauthors Hubert, M. , Debruyne, M. \ Rousseeuw, P J. APACrefauthors \ 2018 . M inimum C ovariance D eterminant and extensions M inimum C ovariance D eterminant and extensions . Wiley Interdisciplinary Reviews: Computational Statistics 10 3 e1421 . APACrefURL https://wires.onlinelibrary.wiley.com/doi/full/10.1002/wics.1421 APACrefURL

    work page doi:10.1002/wics.1421 2018

  8. [8]

    , Rousseeuw, P J

    Hubert:MFC APACrefauthors Hubert, M. , Rousseeuw, P J. \ Segaert, P. APACrefauthors \ 2017 . Multivariate and functional classification using depth and distance Multivariate and functional classification using depth and distance . Advances in Data Analysis and Classification 11 445--466

    work page 2017

  9. [9]

    , Rousseeuw, P J

    Hubert:DetMCD APACrefauthors Hubert, M. , Rousseeuw, P J. \ Verdonck, T. APACrefauthors \ 2012 . A deterministic algorithm for robust location and scatter A deterministic algorithm for robust location and scatter . Journal of Computational and Graphical Statistics 21 3 618--637

    work page 2012

  10. [10]

    \ Van Driessen , K

    Hubert:Discrim APACrefauthors Hubert, M. \ Van Driessen , K. APACrefauthors \ 2004 . Fast and robust discriminant analysis Fast and robust discriminant analysis . Computational Statistics & Data Analysis 45 2 301--320

    work page 2004

  11. [11]

    , Parelius, J

    Liu:Depth APACrefauthors Liu, R. , Parelius, J. \ Singh, K. APACrefauthors \ 1999 . Multivariate analysis by data depth: descriptive statistics, graphics and inference Multivariate analysis by data depth: descriptive statistics, graphics and inference . The Annals of Statistics 27 783--840

    work page 1999

  12. [12]

    , Rousseeuw, P

    robustbase APACrefauthors Maechler, M. , Rousseeuw, P. , Croux, C. , Todorov, V. , R\"uckstuhl, A. \ Salibian-Barrera, M. APACrefauthors \ 2024 . robustbase : Basic robust statistics robustbase : Basic robust statistics \ [ ]. APACrefURL https://CRAN.R-project.org/package=robustbase APACrefURL

    work page 2024

  13. [13]

    APACrefauthors \ 2004

    mclachlan2004discriminant APACrefauthors McLachlan, G J. APACrefauthors \ 2004 . Discriminant Analysis and Statistical Pattern Recognition Discriminant Analysis and Statistical Pattern Recognition \ ( 544). John Wiley & Sons

    work page 2004

  14. [14]

    APACrefauthors \ 2023

    RCT2023 APACrefauthors R Core Team . APACrefauthors \ 2023 . R: A Language and Environment for Statistical Computing R: A language and environment for statistical computing \ [ ]. Vienna, Austria . APACrefURL https://www.R-project.org/ APACrefURL

    work page 2023

  15. [15]

    Survival Regression with Accelerated Failure Time Model in XGBoost

    Raymaekers:Silhouettes APACrefauthors Raymaekers, J. \ Rousseeuw, P J. APACrefauthors \ 2022 . Silhouettes and Quasi Residual Plots for Neural Nets and Tree-Based Classifiers Silhouettes and quasi residual plots for neural nets and tree-based classifiers . Journal of Computational and Graphical Statistics 31 1332--1343 . APACrefURL https://doi.org/10.1080...

    work page doi:10.1080/10618600.2022.2050249 2022

  16. [16]

    \ Rousseeuw, P J

    cellMCD APACrefauthors Raymaekers, J. \ Rousseeuw, P J. APACrefauthors \ 2023 1 . The Cellwise M inimum C ovariance D eterminant Estimator The cellwise M inimum C ovariance D eterminant estimator . Journal of the American Statistical Association . APACrefURL https://www.tandfonline.com/doi/full/10.1080/01621459.2023.2267777 APACrefURL

    work page doi:10.1080/01621459.2023.2267777 2023

  17. [17]

    \ Rousseeuw, P J

    cellWise APACrefauthors Raymaekers, J. \ Rousseeuw, P J. APACrefauthors \ 2023 2 . cellWise : Analyzing Data with Cellwise Outliers cellWise : Analyzing Data with Cellwise Outliers \ [ ]. APACrefURL https://CRAN.R-project.org/package=cellWise APACrefURL

    work page 2023

  18. [18]

    \ Rousseeuw, P J

    classmap_R APACrefauthors Raymaekers, J. \ Rousseeuw, P J. APACrefauthors \ 2023 3 . classmap : Visualizing Classification Results classmap : Visualizing Classification Results \ [ ]. APACrefURL https://CRAN.R-project.org/package=classmap APACrefURL

    work page 2023

  19. [19]

    \ Rousseeuw, P J

    Challenges APACrefauthors Raymaekers, J. \ Rousseeuw, P J. APACrefauthors \ 2024 . Challenges of cellwise outliers Challenges of cellwise outliers . Econometrics and Statistics . APACrefURL https://doi.org/10.1016/j.ecosta.2024.02.002 APACrefURL

    work page doi:10.1016/j.ecosta.2024.02.002 2024

  20. [20]

    , Rousseeuw, P J

    Raymaekers:ClassMap APACrefauthors Raymaekers, J. , Rousseeuw, P J. \ Hubert, M. APACrefauthors \ 2022 . Class Maps for Visualizing Classification Results Class maps for visualizing classification results . Technometrics 64 2 151--165 . APACrefURL https://doi.org/10.1080/00401706.2021.1927849 APACrefURL

    work page doi:10.1080/00401706.2021.1927849 2022

  21. [21]

    APACrefauthors \ 1984

    Rousseeuw:LMS APACrefauthors Rousseeuw, P J. APACrefauthors \ 1984 . L east M edian of S quares Regression L east M edian of S quares regression . Journal of the American Statistical Association 79 388 871--880

    work page 1984

  22. [22]

    APACrefauthors \ 1985

    Rousseeuw:MCD APACrefauthors Rousseeuw, P J. APACrefauthors \ 1985 . Multivariate estimation with high breakdown point Multivariate estimation with high breakdown point . W. Grossmann, G. Pflug, I. Vincze \ W. Wertz\ ( ), Mathematical Statistics and Applications, Vol. B Mathematical Statistics and Applications, Vol. B \ ( \ 283--297). Reidel

    work page 1985

  23. [23]

    APACrefauthors \ 1987

    Silh1987 APACrefauthors Rousseeuw, P J. APACrefauthors \ 1987 . Silhouettes: a Graphical Aid to the Interpretation and Validation of Cluster Analysis Silhouettes: a graphical aid to the interpretation and validation of cluster analysis . Journal of Computational and Applied Mathematics 20 388 53--65 . APACrefURL https://doi.org/10.1016/0377-0427(87)90125-...

    work page doi:10.1016/0377-0427(87)90125-7 1987

  24. [24]

    \ Van Driessen , K

    Rousseeuw:FastMCD APACrefauthors Rousseeuw, P J. \ Van Driessen , K. APACrefauthors \ 1999 . A Fast Algorithm for the M inimum C ovariance D eterminant Estimator A fast algorithm for the M inimum C ovariance D eterminant estimator . Technometrics 41 3 212--223

    work page 1999

  25. [25]

    APACrefauthors \ 2023

    rrcov APACrefauthors Todorov, V. APACrefauthors \ 2023 . rrcov : Scalable Robust Estimators with High Breakdown Point rrcov : Scalable Robust Estimators with High Breakdown Point \ [ ]. APACrefURL https://CRAN.R-project.org/package=rrcov APACrefURL

    work page 2023

  26. [26]

    , Raymaekers, J

    Vranckx:RT-RQDA APACrefauthors Vranckx, I. , Raymaekers, J. , De Ketelaere , B. , Rousseeuw, P J. \ Hubert, M. APACrefauthors \ 2021 . Real-time discriminant analysis in the presence of label and measurement noise Real-time discriminant analysis in the presence of label and measurement noise . Chemometrics and Intelligent Laboratory Systems 208 104197

    work page 2021

  27. [27]

    , De Ketelaere , B

    Wouters:floral APACrefauthors Wouters, N. , De Ketelaere , B. , Deckers, T. , Baerdemaeker, J D. \ Saeys, W. APACrefauthors \ 2015 . Multispectral detection of floral buds for automated thinning of pear Multispectral detection of floral buds for automated thinning of pear . Computers and Electronics in Agriculture 113 93--103

    work page 2015