REVIEW 1 major objections 3 minor 34 references
Ultrahigh-dimensional Quadratic Discriminant Analysis Using Random Projections
T0 review · 1 major / 3 minor · reviewed 2026-08-07 · deepseek-v4-flash
Pith's one-line read The paper proves that averaging QDA over many random low-dimensional projections drives the misclassification probability to zero as the dimension grows, even when the number of variables far exceeds the sample size.
desk verdict Worth a serious referee: the d/p scaling with KL divergence is a real insight, but the sample-version theorem rests on a deferred proof that must handle singular sample covariances. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The object doing the work is the RPE-QDA discriminant function $D^{{RPE}}$_{k',k}(Z) = $B^{{-1}}$ Σ_{b=1}^B D^b_{k',k}(Z), the average of B QDA log-discriminants computed after projecting the data with independent i.i.d. standard Gaussian d×p random matrices. The load-bearing estimate is Lemma 2, which bounds the expected projected precision matrix Ψ = E_R[R^T(RΣR^T)^{-1}R] between (d/(p λ_max(Σ))) I and (d/(p λ_min(Σ))) I; this squeezes the averaged discriminant toward a quantity comparable to (d/p)·KL_{k,k'}, converting the ensemble of projections into a scaled version of the KL divergence and yielding the perfect-classification theorems.
What would settle it
Simulate two Gaussian classes with equal means and covariance matrices that differ only on a block of p^α coordinates with α<1, so the KL divergence grows like p^α and assumption (A3) fails. If RPE-QDA's error still converges to zero as p grows, the linear-KL condition is stronger than needed; if the error stays bounded away from zero, assumption (A3) is the load-bearing separation requirement.
Extended reading notes
Core claim
On its own terms, the paper claims that under assumptions (A1)-(A3), namely controlled growth of the means, covariance spectra with at most p^ξ eigenvalues deviating from a common baseline and no eigenvalue growing faster than p^α for α<1, and KL divergence between every pair of classes growing at least linearly in p, the RPE-QDA classifier satisfies lim_{p→∞} lim_{B→∞} $Δ^{{RPE-QDA}}$ = 0 (Theorem 2(c)). For the sample version, Theorem 3 gives lim_{min{n,p}→∞} lim_{B→∞} $Δ^{{RPE-QDA}}$_n = 0 under the additional balanced-design and projection-dimension condition (A4). The companion structural result, highlighted in Remark 1 and Section 3.2, is that the classical QDA discriminant is asymptotically equivalent to the KL divergence after division by p, whereas the RPE-QDA discriminant, after division by d, is asymptotically equivalent to the same KL divergence scaled by d/p; this quantifies the information loss induced by projection and is traded off against a large computational saving.
Load-bearing premise
The guarantee collapses if the true class separation, measured by Kullback-Leibler divergence, does not grow at least in proportion to the dimension p; separation concentrated in a few coordinates is not enough for the theorem, even if a good classifier exists.
Editorial extensions
If this is right
- When (A1)-(A3) hold and p grows sub-exponentially in n, RPE-QDA drives the misclassification probability to zero asymptotically; the user only needs projection dimension d = O(log p) and a sufficiently large ensemble size B.
- The method stays consistent when class means are identical and only covariance matrices differ, a case where trace-based and distance-based competitors degrade to random guessing (Schemes 2 and 4 of the simulations).
- Computational cost drops from order O(nmax p^2 ∨ p^3) for classical QDA to O(B(d p nmax + d^3)), and the B projection steps are embarrassingly parallel.
- The discriminant's d/p scaling with KL divergence quantifies how much discriminative power is lost per projection: accuracy improves as d grows, up to the sample-size limit.
- Plugging in sample means and covariances costs no extra asymptotic loss: the sample RPE-QDA classifier retains the same d/p-scaled KL structure (Remark 2).
Reading between the lines
- A direct consequence the authors leave implicit: the d/p factor gives a practical rule of thumb — when KL/p is only barely above the threshold ν0, the projection dimension d must be a non-negligible fraction of p to keep the scaled margin positive, so small-d ensembles will be reliable mainly when separation is diffuse across coordinates.
- The theory suggests a testable extension to non-Gaussian classes with comparable log-density concentration: the same averaging should approximate a scaled divergence whenever the projected log-likelihood ratios concentrate, which the real-data experiments hint at but do not prove.
- Pairing random-projection ensembles with variable screening, as the authors mention in the Discussion, could relax (A3): concentrating KL into a screened subvector would raise the effective per-coordinate divergence while keeping the d/p dilution small.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes RPE-QDA, a random-projection ensemble version of quadratic discriminant analysis for ultrahigh-dimensional classification. The method averages QDA discriminant functions computed in B random d-dimensional subspaces, avoiding inversion of the p×p sample covariance. The main theoretical results are Theorems 1–3: under assumptions (A1)–(A4), the population QDA rule, the population RPE-QDA rule, and the sample RPE-QDA rule all achieve perfect classification in suitable asymptotic limits. The paper also quantifies the information loss of random projection by showing that the RPE-QDA discriminant is asymptotically equivalent to a scaled KL divergence with scaling factor d/p (Remark 1, Theorem 2). The methodology is evaluated in four simulation schemes and four gene-expression data sets, with comparisons to several existing high-dimensional classifiers.
Significance. If the results hold, the paper makes a useful contribution: it offers a computationally inexpensive QDA procedure that can separate populations differing in covariance structure even when means, traces, and other low-dimensional summary measures are identical, a setting where several existing methods fail (Schemes 2 and 4). The d/p scaling of the discriminant is a clean and falsifiable quantification of the cost of random projection. The paper also provides code links and extensive numerical comparisons. However, the sample-version theorem currently rests on a deferred lemma and on an application of Lemma 2 to singular sample covariances, which is not justified in the submitted text; this is the main obstacle to accepting the theoretical claims as they stand.
major comments (1)
- [Section 7.1, proof of Theorem 3, and Lemma 4] The proof of Theorem 3 reduces the first term in equation (15) to a conditional SLLN for the sequence bD^{R_b}(Z) given (Z, X_n, Y_m), saying this follows 'similar to the proof of Theorem 2'. That argument uses Lemma 2, whose bound on E_R[R^T(RΣR^T)^{-1}R] is stated only for positive-definite Σ. For p > n_k, the sample covariance bΣ_k is singular with rank n_k − 1, so λ_min(bΣ_k) = 0 and Lemma 2 does not apply. This is not a purely technical gap: for a fixed singular bΣ of rank r, the conditional expectation can be infinite unless r > d+1. For example, with p=3, d=1, and bΣ = diag(1,1,0), the (3,3) entry of E_R[R^T(R bΣ R^T)^{-1}R] equals E[r_3^2/(r_1^2+r_2^2)] = E[r_3^2] E[1/χ_2^2] = ∞, so the projected quadratic form has infinite conditional mean and the SLLN is not available. Assumption (A4) only imposes d = o(n_min); it does not ensure n_min > d+2 for every fixed n at which the inner B→∞ limit is taken. Unless Lemma 4 in the supplementary supplies a genuinely different singular-case argument, Theorem 3 is not proved as written. A concrete fix would be to strengthen (A4) to n_min > d+2 for all n in the sequence or to provide a truncated-moment argument that avoids infinite conditional expectations.
minor comments (3)
- [Section 1.2, Notations] The definitions a ∧ b = max{a,b} and a ∨ b = min{a,b} are reversed relative to standard usage and are inconsistent with the later definitions of n_min = ∧_k n_k and n_max = ∨_k n_k; this should be corrected.
- [Section 7.1, proof of Theorem 2] The sentence 'as RR^T ∼ Wishart(I_p, p)' has a dimension mismatch: R is d × p, so RR^T is a d × d Wishart matrix with p degrees of freedom, not Wishart(I_p, p).
- [Theorem 2(b) and Lemma 3] The notation 'p^{-1}KL_{k,k'}{1+o(1)}' inside a probability statement is ambiguous: the o(1) term should be made explicit as deterministic or as holding with probability tending to one uniformly in the relevant random variables.
Circularity Check
No significant circularity: the RPE-QDA derivation is self-contained, with the d/p scaling derived from Gaussian random matrix expectations rather than from fitted inputs.
full rationale
The paper's central derivation chain is input-to-output. Theorem 1 shows that under assumptions (A1)-(A3), the classical QDA discriminant satisfies p^{-1}|D(Z) - KL| -> 0 in probability via a variance calculation and Chebyshev's inequality, so that KL/p >= nu0 yields vanishing misclassification. Theorem 2 derives the RPE-QDA population discriminant as the R-expectation of the projected discriminant and obtains d^{-1}D^{RP} approximately equal to p^{-1}KL through the Gaussian random matrix property encoded in Lemma 2, i.e., E_R[R^T(R Sigma R^T)^{-1}R] is bounded by (d/p) times inverse-covariance-type quantities; this scaling factor is derived, not fitted. Theorem 3 conditions on the training data and invokes Lemma 4 to show the sample version inherits the same margin; the proof is deferred to the supplementary but this is a verification gap, not a circular reduction. No parameter is fit to a subset of data and then renamed as a prediction: d and B are user-chosen tuning parameters, and the theory is stated for population quantities and sample analogs without estimating the KL divergence itself for the theoretical guarantees. Assumption (A3) is a sufficient separability condition, not a definition of the misclassification probability; the theorem that KL/p >= nu0 implies error -> 0 requires the variance-concentration argument and is not a tautology. The only self-citation, Mukhopadhyay and Dunson (2020), appears in the introduction and in the Discussion as a pointer to related targeted random projection work and as a future research direction; it is not load-bearing for any theorem. The skeptic's concern about singular sample covariances in the proof of Theorem 3 is a possible correctness or assumption gap, but it is not an instance of the paper's claims reducing to their own inputs. Accordingly, no circular step is identified.
Assumptions & free parameters
free parameters (2)
- d =
10 (simulations), 2 or n_min ∧ ⌈log p⌉ (real data)
- B =
200
assumptions (6)
- domain assumption Gaussianity of class populations
- domain assumption Assumption (A1): max_k ||μ_k||^2 = O(p)
- domain assumption Assumption (A2): covariance eigenstructure with at most O(p^ξ) non-minimal eigenvalues and max eigenvalue O(p^α), α<1
- domain assumption Assumption (A3): liminf p^{-1} min KL ≥ ν0 > 0
- domain assumption Assumption (A4): balanced sample sizes and d = o(n_min), d = O(log p)
- ad hoc to paper Lemma 2 eigenvalue bounds on E[R^T(RΣR^T)^{-1}R]
Cite this review
Pith. "Pith review of Ultrahigh-dimensional Quadratic Discriminant Analysis Using Random Projections." pith.science (2026). https://pith.science/paper/RNEZZRT7
@misc{pith2026250523324,
author = {Pith},
title = {Pith review of: Ultrahigh-dimensional Quadratic Discriminant Analysis Using Random Projections},
year = {2026},
howpublished = {\url{https://pith.science/paper/RNEZZRT7}},
note = {Machine review of arXiv:2505.23324}
}
abstract
This paper investigates the effectiveness of using the Random Projection Ensemble (RPE) approach in Quadratic Discriminant Analysis (QDA) for ultrahigh-dimensional classification problems. Classical methods such as Linear Discriminant Analysis (LDA) and QDA are used widely, but face significant challenges in their implementation when the data dimension (say, $p$) exceeds the sample size (say, $n$). In particular, both LDA (using the Moore-Penrose inverse for covariance matrices) and QDA (even with known covariance matrices) may perform as poorly as random guessing when $p/n \to \infty$ as $n \to \infty$. The RPE method, known for addressing the curse of dimensionality, offers a fast and effective solution without relying on selective summary measures of the competing distributions. This paper demonstrates the practical advantages of employing RPE on QDA in terms of classification performance as well as computational efficiency. We establish results for limiting perfect classification in both the population and sample versions of the proposed RPE-QDA classifier, under fairly general assumptions that allow for sub-exponential growth of $p$ relative to $n$. Several simulated and gene expression data sets are analyzed to evaluate the performance of the proposed classifier in ultrahigh-dimensional~scenarios.
Figures
Reference graph
Works this paper leans on
-
[1]
Achlioptas, D. (2003). Database-friendly random projections: J ohnson- L indenstrauss with binary coins. Journal of Computer and System Sciences , 66(4):671--687
2003
-
[2]
Ahfock, D. C., Astle, W. J., and Richardson, S. (2021). Statistical properties of sketching algorithms. Biometrika , 108(2):283--297
work page 2021
-
[3]
Aoshima, M. and Yata, K. (2014). A distance-based, misclassification rate adjusted classifier for multiclass, high-dimensional data. Annals of the Institute of Statistical Mathematics , 66(5):983--1010
work page 2014
-
[4]
Aoshima, M. and Yata, K. (2019). Distance-based classifier by data transformation for high-dimension, strongly spiked eigenvalue models. Annals of the Institute of Statistical Mathematics , 71(3):473--503
work page 2019
-
[5]
Ayyala, D. N., Ghosh, S., and Linder, D. F. (2022). Covariance matrix testing in high dimension using random projections. Computational Statistics , 37(3):1111--1141
work page 2022
-
[6]
Bickel, P. J. and Levina, E. (2004). Some theory of F isher's linear discriminant function, `naive B ayes', and some alternatives when there are many more variables than observations. Bernoulli , 10(6):989--1010
work page 2004
-
[7]
Bouveyron, C., Girard, S., and Schmid, C. (2007). High-dimensional discriminant analysis. Communications in Statistics. Theory and Methods , 36(13-16):2607--2623
work page 2007
-
[8]
Cannings, T. I. and Samworth, R. J. (2017). Random-projection ensemble classification. Journal of the Royal Statistical Society. Series B. Statistical Methodology , 79(4):959--1035
work page 2017
Show all 34 references
-
[9]
Dasgupta, S. (1999). Learning mixtures of G aussians. In 40th A nnual S ymposium on F oundations of C omputer S cience ( N ew Y ork, 1999) , pages 634--644. IEEE Computer Society, Los Alamitos, CA
1999
-
[10]
and Gupta, A
Dasgupta, S. and Gupta, A. (2003). An elementary proof of a theorem of J ohnson and L indenstrauss. Random Structures & Algorithms , 22(1):60--65
2003
-
[11]
C., Costa, I
De Souto, M. C., Costa, I. G., De Araujo, D. S., Ludermir, T. B., and Schliep, A. (2008). Clustering cancer gene expression data: a comparative study. BioMed Central Bioinformatics , 9:1--14
2008
-
[12]
and Bostrom, H
Deegalla, S. and Bostrom, H. (2006). Reducing high-dimensional data by principal component analysis vs. random projection for nearest neighbor classification. In International Conference on Machine Learning and Applications, 2006 , pages 245--250. Institute of Electrical and E...
2006
-
[13]
Dudoit, S., Fridlyand, J., and Speed, T. P. (2002). Comparison of discrimination methods for the classification of tumors using gene expression data. Journal of the American Statistical Association , 97(457):77--87
2002
-
[14]
and Kab \'a n, A
Durrant, R. and Kab \'a n, A. (2013). Sharp generalization error bounds for randomly-projected classifiers. In International Conference on Machine Learning , pages 693--701. PMLR
2013
-
[15]
Durrant, R. J. and Kab\'an, A. (2015). Random projections as regularizers: learning a linear discriminant from fewer observations than dimensions. Machine Learning , 99(2):257--286
2015
-
[16]
Fan, Y., Kong, Y., Li, D., and Zheng, Z. (2015). Innovated interaction screening for high-dimensional nonlinear classification. The Annals of Statistics , 43(3):1243--1272
2015
-
[17]
Fern, X. Z. and Brodley, C. E. (2003). Random projection for high dimensional data clustering: A cluster ensemble approach. In International Conference on Machine Learning , pages 186--193
2003
-
[18]
Friedman, J. H. (1989). Regularized discriminant analysis. Journal of the American Statistical Association , 84(405):165--175
1989
-
[19]
Gataric, M., Wang, T., and Samworth, R. J. (2020). Sparse principal component analysis via axis-aligned random projections. Journal of the Royal Statistical Society. Series B. Statistical Methodology , 82(2):329--359
2020
-
[20]
Heckel, R., Tschannen, M., and B\"olcskei, H. (2017). Dimensionality-reduced subspace clustering. Information and Inference. A Journal of the Institute of Mathematics and its Applications , 6(3):246--283
2017
-
[21]
Jiang, B., Wang, X., and Leng, C. (2018). A direct approach for sparse quadratic discriminant analysis. Journal of Machine Learning Research , 19:Paper No. 31, 37
2018
-
[22]
Johnson, W. B. and Lindenstrauss, J. (1984). Extensions of L ipschitz mappings into a H ilbert space. In Conference in Modern Analysis and Probability , volume 26 of Contemporary Mathematics , pages 189--206. American Mathematical Society, Providence, RI
1984
-
[23]
Johnstone, I. M. (2001). On the distribution of the largest eigenvalue in principal components analysis. The Annals of Statistics , 29(2):295--327
2001
-
[24]
Klanke, S., Vijayakumar, S., and Schaal, S. (2008). A library for locally weighted projection regression. Journal of Machine Learning Research , 9:623--626
2008
-
[25]
and Leibler, R
Kullback, S. and Leibler, R. A. (1951). On information and sufficiency. Annals of Mathematical Statistics , 22:79--86
1951
-
[26]
J., and Church, K
Li, P., Hastie, T. J., and Church, K. W. (2006). Very sparse random projections. In Proceedings of the 12th ACM SIGKDD , pages 287--296
2006
-
[27]
and Shao, J
Li, Q. and Shao, J. (2015). Sparse quadratic discriminant analysis for high dimensional data. Statistica Sinica , 25(2):457--473
2015
-
[28]
Lopes, M., Jacob, L., and Wainwright, M. J. (2011). A more powerful two-sample test in high dimensions using random projection. Advances in Neural Information Processing Systems , 24
2011
-
[29]
L., Tucci, G
Marzetta, T. L., Tucci, G. H., and Simon, S. H. (2011). A random matrix-theoretic approach to handling singular covariance estimates. IEEE Transactions on Information Theory , 57(9):6256--6271
2011
-
[30]
and Dunson, D
Mukhopadhyay, M. and Dunson, D. B. (2020). Targeted random projection for prediction from high-dimensional features. Journal of the American Statistical Association , 115(532):1998--2010
2020
-
[31]
and Kab\'an, A
Palias, E. and Kab\'an, A. (2023). The effect of intrinsic dimension on the B ayes-error of projected quadratic discriminant classification. Statistics and Computing , 33(4):Paper No. 87, 17
2023
-
[32]
Srivastava, R., Li, P., and Ruppert, D. (2016). R APTT : an exact two-sample test in high dimensions using random projections. Journal of Computational and Graphical Statistics , 25(3):954--970
2016
-
[33]
Wu, Y., Qin, Y., and Zhu, M. (2019). Quadratic discriminant analysis for high-dimensional data. Statistica Sinica , 29(2):939--960
2019
-
[34]
Yan, D., Wang, Y., Wang, J., Wang, H., and Li, Z. (2019). K-nearest neighbor search by random projection forests. IEEE Transactions on Big Data , 7(1):147--157
2019
Reviewed August 7, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.