Detection of Signals in Colored Noise: Roy's Largest Root Test for Non-central $F$-matrices

Jamie Evans; Merouane Debbah; Prathapasinghe Dharmawansa; Saman Atapattu

REVIEW 3 major objections 4 minor 78 references

Detection of Signals in Colored Noise: Roy's Largest Root Test for Non-central $F$-matrices

T0 review · 3 major / 4 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This paper derives a closed-form cumulative distribution function for Roy's largest root, the leading eigenvalue of a non-central F-matrix with rank-one non-centrality, and uses it to analyze detection performance in colored noise with…

desk verdict A genuinely new CDF for Roy's largest root under rank-one non-centrality, but Theorem 3 as printed has a variable-conversion typo (2/(t+1) should be 2/t+1) that fails an m=1 sanity check; the method is sound and the paper deserves referee time after a fix. read the letter →

arxiv 2412.05306 v1 pith:C5TN2XMU submitted 2024-11-27 eess.SP cs.ITmath.IT

Prathapasinghe Dharmawansa , Saman Atapattu , Jamie Evans , Merouane Debbah This is my paper · ORCID

classification eess.SPcs.ITmath.IT MSC 62H1060B2062H15

keywords non-centralF-matrixRoy'slargestrootsignaldetectioncolorednoiserandommatrixtheoryorthogonalpolynomialsreceiveroperatingcharacteristicshigh-dimensionalasymptotics

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper targets the problem of detecting a non-random signal in colored noise when the noise covariance is unknown, using the leading eigenvalue of the whitened sample covariance matrix (Roy's largest root) as the test statistic. Its central result is an exact finite-dimensional CDF for that statistic: when the non-centrality has rank one, the CDF of the largest eigenvalue of a non-central F-matrix is a closed-form determinant whose size depends only on n−m, the excess of noise-only samples over the system dimension. This makes the n=m case a simple exponential expression. The CDF yields explicit receiver operating characteristic (ROC) curves and, in the high-dimensional limit, shows that the test has power only above a phase-transition SNR, with the required SNR scaling differing between fixed and growing dimensions. A sympathetic reader would care because it turns an eigenvalue detection statistic previously studied through asymptotics or simulation into an exactly computable quantity.

What carries the argument

The argument combines the zonal-polynomial joint eigenvalue density of a non-central F-matrix with the orthogonal-polynomial method for random matrix integrals. A change of variables x = λ/(1+λ) converts the eigenvalue density into a Jacobi-type weight, after which the m-fold integral for the CDF is evaluated via the Andréief–Heine (Cauchy–Binet) identity, yielding a determinant of size α+1 = n−m+1. A contour-integral representation of the confluent hypergeometric function re-sums the first column of the determinant into the compact Φ_i factors. This determinant dimension depends only on n−m, so computational cost tracks the quality of the noise-covariance estimate rather than the ambient dimension m, and the simplest case n=m collapses to a scalar exponential.

What would settle it

For the smallest non-trivial case α=1, m=2, numerically evaluate both sides of the identity connecting (101) and (103), first with s1 and s2 distinct and then with s1→y, s2→x; if the limit does not reproduce the determinant ratio on the right of (103), Theorem 3 is unsupported. Alternatively, simulate F = $S^{{-1/2}}$$RS^{{-1/2}}$ with m=2, n=3, p=5, ω=2 and compare the empirical CDF of λmax with the closed form in (23).

Watch

Extended reading notes

Core claim

The paper claims that for independently distributed R ∼ CW_m(p, Σ, Θ) and S ∼ CW_m(n, Σ) with rank-one non-centrality Θ, the CDF of the largest eigenvalue λmax of F = $S^{{-1/2}}$$RS^{{-1/2}}$ is $F^{{(α)}}$_{λmax}(t;ω) = K(α,β,m) $e^{{-ω/(1+t)}}$ (t/(1+t))^{m(α+β+m)} det[Φ_i(t,ω) Ψ_{i,j}(t)], where α = n−m, β = p−m, the determinant is of size α+1, and the special case α=0 (n=m) degenerates to $e^{{-ω/(1+t)}}$ (t/(1+t))^{m(β+m)}. The same theorem drives the ROC analysis in Section IV, including the high-dimensional phase transition: for m/p→c1 and m/n→c2∈(0,1) with SNR = pγ, the test has asymptotic power only when γ exceeds the critical value γp = (c2+r)/(1−c2) with r = √(c1+c2−c1c2). When m=n, the required scaling changes: power is retained in the high-dimensional limit only for SNR of order O($p^{2}$), whereas for fixed m and n the requirement is SNR of order O(p).

Load-bearing premise

The load-bearing premise is that the 0/0 limiting argument in Appendix D, which coalesces several distinct parameters s_i into a single value inside an Andréief–Heine determinant, is valid; this step is asserted by reference to two earlier works rather than proved, and the main determinant formula collapses if that limit is not legitimate.

Editorial extensions

If this is right

For any finite m, n, p with p,n ≥ m and rank-one non-centrality, the detection and false-alarm probabilities of Roy's largest-root test can be computed exactly from the determinant formula, with no simulation needed.
When n = m, the ROC collapses to the explicit closed form PD = 1 − (1−PF) exp{−ω(1 − [1−PF]^{1/[m(β+m)]})}, which is an achievable lower bound on all ROC profiles for fixed other parameters.
In the high-dimensional regime with m/p→c1 and m/n→c2∈(0,1), the largest-root test has no asymptotic detection power for SNR = pγ below the phase-transition threshold γp, and power tends to 1 above it.
For m = n, the required SNR scaling to retain power is O(p) in the fixed-dimensional setting but O(p^2) in the high-dimensional setting, and for m < n with SNR = O(p) the leading eigenvalue cannot detect weak signals asymptotically.
The test has the constant-false-alarm-rate (CFAR) property under H0, since the null CDF is independent of the noise covariance Σ.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The determinant structure, whose size is n−m+1, suggests that the same orthogonal-polynomial machinery could produce exact distributions for other spectral statistics of non-central F-matrices—such as the smallest eigenvalue or the trace—by adapting the determinant entries while keeping the dimension fixed by n−m.
Because the rank-one non-centrality is the building block, higher-rank non-centrality parameters might be treated by writing the CDF as a mixture or by a perturbation expansion around the rank-one case, although the paper explicitly leaves arbitrary rank as an open problem.
The phase-transition finding implies that any eigenvalue-based detector in this colored-noise setting inherits a fundamental detection boundary: below the critical SNR the largest root carries no information asymptotically, which could serve as a design target for radar systems that must work with limited noise-only samples.
A natural testable extension is to differentiate the determinant CDF to obtain the exact density of Roy's largest root, which would allow likelihood-ratio-based threshold calibration and finite-sample corrections to Tracy–Widom approximations.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Quick take: the paper has a real, useful result—an exact CDF for the largest eigenvalue of a rank-one non-central F-matrix that costs a determinant of size n−m+1, not m—but the printed Theorem 3 contains a variable-conversion typo that makes the formula wrong as written. The stress-test is right: the Jacobi argument should be 2/t+1, not 2/(t+1). I checked the m=1, α=1, β=0, ω=0 case: the printed Corollary 3 gives 2t²/(1+t)³, which fails at t=1 (1/4 vs 3/4) and tends to 0 instead of 1. The corrected argument reproduces the exact scalar CDF. So the core method is sound; the theorem as stated is not.

What's genuinely new: the α-dimensional determinantal representation (Theorem 3) is a real improvement over the authors' own ISIT version, and the special cases (n=m, ω=0) reduce to clean formulas. The proof follows a known orthogonal-polynomial/contour-integral route and is detailed enough to be checked. The ROC analysis, especially the m=n case and the O(p) vs O(p²) SNR phase transition, is a nice addition.

Soft spots: besides the typo, Section IV overclaims the ROC monotonicity equivalence in Eqs. (36)–(38)—that's a one-way implication for fixed signal direction, not a full equivalence. The singular F relabeling (30) is asserted without proof; it may be true but needs a reference or a short argument. The limiting procedure in Appendix D that coalesces the s_i parameters is cited to [12],[56] rather than proved; that's acceptable for a paper of this type, but a referee should push for a few more details.

Bottom line: this paper deserves a serious referee. The main theorem is valuable and likely correct after the fix, but nobody should rely on Eq. (23) in its current form. I would not cite v1; I'd cite the corrected version. For the reading group: worth bringing because it's a clean example of how a localized typo can masquerade as a failed result.

Referee Report

3 major / 4 minor

Summary. This paper studies detection of a non-random signal in colored Gaussian noise with unknown covariance, using Roy's largest root of the whitened sample covariance matrix as the test statistic. The main theoretical contribution is an exact finite-dimensional CDF, Theorem 3 (Eq. 23), for the largest eigenvalue of a complex non-central F-matrix with a rank-one noncentrality parameter, derived via orthogonal polynomials and contour integrals; the determinant size depends on α = n − m, yielding a scalar formula for m = n (Corollary 2) and a reduction to the central F case (Corollary 3). The authors use this CDF to derive ROC expressions, the CFAR property, finite-dimensional monotonicity results, and high-dimensional detection limits, including the sub- and super-critical SNR phase transition.

Significance. If the main CDF formula is corrected, the paper would provide a valuable exact finite-dimensional characterization of Roy's largest root under a rank-one noncentrality, with the determinant dimension controlled by n − m rather than by m, enabling efficient evaluation when n − m is small. The α = 0 closed form and its high-dimensional consequences (an O(p²) SNR requirement when m = n) are interesting, and Proposition 1 supplies a matrix-integral identity of independent utility. However, Theorem 3 as printed contains a variable-conversion error in the Jacobi-polynomial argument and fails a scalar sanity check, so the quantitative ROC curves and corollaries depending on general α are not valid as stated. The underlying method appears sound and the error is localized, but the paper's significance will be realized only after the printed formulas are corrected and re-verified.

major comments (3)

[Section III.B, Theorem 3 / Corollary 3] The argument of the Jacobi polynomial in Eq. (23) is wrong as printed. Appendix D produces an expression in the x-threshold t_x with Jacobi argument 2/t_x − 1; substituting t_x = t/(1+t) via Eq. (21) gives 2/t + 1, not 2/(t+1). As a decisive check, take m = p = 1, n = 2 (α = 1, β = 0) and ω = 0: the exact CDF is 1 − (1+t)⁻², whereas Corollary 3 with the printed Ψ₂,₂ = 2/(t+1) gives 2t²/(1+t)³, which tends to 0 as t → ∞ and equals 1/4 at t = 1 instead of 3/4. Replacing 2/(t+1) by 2/t + 1 reproduces the exact scalar CDF. The same incorrect argument appears in Eqs. (68), (70), (71), and (95)/Proposition 1; these should be corrected consistently, and the numerical figures and finite-dimensional ROC formulas (32)–(33) should be re-generated or verified with the corrected formula.
[Appendix D, Eqs. (100)–(103)] The coalescing limit s₁ → y, s₂, ..., s_{α+1} → x is asserted by reference to [12], [56] rather than proved. Because both the numerator determinant and the Vandermonde denominator vanish in this limit, the resulting derivative structure and prefactor are load-bearing for Theorem 3. Please provide a self-contained derivation or a precise statement of the identity being invoked.
[Section IV.A, Eq. (36)] The displayed equivalence between the ROC inequalities and the Loewner order is overclaimed. For a fixed vector a, P_D depends on Σ only through the scalar aᴴΣ⁻¹a, so the Loewner order is sufficient but not necessary for the quadratic-form inequalities; many matrices that are not Loewner-comparable yield the same ordering. The direction used for the majorization conclusion (Loewner order ⇒ ROC order) is correct and should be stated as an implication, or the equivalence should be formulated over all vectors a.

minor comments (4)

[Fig. 3 caption] The limiting curves in the two panels appear to be swapped: for ω = 2 (O(1)) the limit should be e^{−5/t}, and for ω = p it should be e^{−6/t}.
[Fig. 8 caption] The sentence 'The ROC profiles ... are depicted in Fig. 8a' should refer to Fig. 8b.
[After Eq. (58)] The phrase 'we the following bounds are in order' is missing a verb; it should read 'we have the following bounds' or 'the following bounds are in order'.
[Section III.B, Eq. (30)] The relabeling rule for the singular F-matrix case (p < m) is stated without derivation or a specific reference; please justify it or provide a precise citation beyond the joint density result in [37].

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the CDF derivation is input-to-output from an external joint density and orthogonal-polynomial identities, with no fitted parameters and an independent ω=0 benchmark.

full rationale

The paper's central claim, Theorem 3, is a closed-form CDF for the largest eigenvalue of a non-central F-matrix with rank-one non-centrality. The derivation is not circular: it starts from the joint eigenvalue density of the non-central F-matrix stated in Theorem 2 and credited to James [37], then evaluates the distribution of the maximum using standard orthogonal-polynomial, contour-integral, and determinant identities. The key determinant evaluation in Appendix D is based on Mehta's integral identity [53], with the coalescing-parameter limiting argument attributed to external sources [12], [56]; although [12] involves one of the present authors as a coauthor, the cited identity is a standard mathematical result with independent standing, and no target conclusion is assumed. No parameter is fitted to data, and the ω=0 specialization is cross-checked against the previously published central F CDF [4, Corollary 8], which is an external benchmark. The self-references that appear ([1], [17], [20]) are not used to supply the finite-dimensional CDF: [1] is a prior alternative expression with a larger determinant that is explicitly compared but not relied upon, and [20] supplies the high-dimensional phase-transition and Gaussian-fluctuation laws as separate mathematical results. The skeptic's observation about the Ψ_{i,j}(t) argument (2/(t+1) versus 2/t+1) is a possible mathematical correctness or typographical issue in the printed formula, not a circularity reduction: the derivation chain still proceeds from external density and polynomial identities to the claimed expression, so it is not equivalent to its own input by construction. For these reasons, the paper exhibits no significant circularity.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper introduces no adjustable fitted parameters and no new theoretical entities. The central CDF is derived from established joint eigenvalue density and orthogonal polynomial identities. The high-dimensional conclusions rest on external phase-transition laws from the cited literature, which are treated as axioms.

assumptions (5)

standard math Complex non-central F-matrix joint eigenvalue density (Theorem 2, Eq. 15) from James [37] is exact for p,n ≥ m.
This is the starting point of the CDF derivation; cited from [37].
domain assumption The non-centrality parameter matrix Ω = ω u u^H has rank one (Section II).
Follows from the rank-one signal model x_i = a s_i; it is load-bearing because Eq. (13) reduces the matrix hypergeometric function to a scalar hypergeometric only for rank-one argument.
standard math Mehta's integral identity (Eq. 100, [53, eqs. 22.4.2, 22.4.11]) and the coalescing-parameter limit (Eqs. 102-103) hold.
Used in Appendix D to collapse the m-dimensional integral to an (α+1)-by-(α+1) determinant.
standard math The contour integral representation of the confluent hypergeometric function (Definition 4, [59]) is valid.
Used in the proof of Theorem 3 to re-sum series and remove the pole at ω=0.
domain assumption Spiked F-matrix phase transition threshold and Tracy-Widom/Gaussian fluctuation laws from [20], [45] hold in the stated asymptotic regime.
Section IV.B assumes these external asymptotic results to derive detection power conclusions.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Detection of Signals in Colored Noise: Roy's Largest Root Test for Non-central $F$-matrices." pith.science (2026). https://pith.science/paper/C5TN2XMU

@misc{pith2026241205306,
  author       = {Pith},
  title        = {Pith review of: Detection of Signals in Colored Noise: Roy's Largest Root Test for Non-central $F$-matrices},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/C5TN2XMU}},
  note         = {Machine review of arXiv:2412.05306}
}

abstract

This paper investigates the signal detection problem in colored noise with an unknown covariance matrix. In particular, we focus on detecting a non-random signal by capitalizing on the leading eigenvalue (a.k.a. Roy's largest root) of the whitened sample covariance matrix as the test statistic. To this end, the whitened sample covariance matrix is constructed via $m$-dimensional $p $ plausible signal-bearing samples and $m$-dimensional $n $ noise-only samples. Since the signal is non-random, the whitened sample covariance matrix turns out to have a {\it non-central} $F$-distribution with a rank-one non-centrality parameter. Therefore, the performance of the test entails the statistical characterization of the leading eigenvalue of the non-central $F$-matrix, which we address by deriving its cumulative distribution function (c.d.f.) in closed-form by leveraging the powerful orthogonal polynomial approach in random matrix theory. This new c.d.f. has been instrumental in analyzing the receiver operating characteristic (ROC) of the detector. We also extend our analysis into the high dimensional regime in which $m,n$, and $p$ diverge such that $m/n$ and $m/p$ remain fixed. It turns out that, when $m=n$ and fixed, the power of the test improves if the signal-to-noise ratio (SNR) is of at least $O(p)$, whereas the corresponding SNR in the high dimensional regime is of at least $O(p^2)$. Nevertheless, more intriguingly, for $m<n$ with the SNR of order $O(p)$, the leading eigenvalue does not have power to detect {\it weak} signals in the high dimensional regime.

Figures

Figures reproduced from arXiv: 2412.05306 by the authors.

**Figure 2.** The effect of ω on the CDF for m = 10, n = 12, and p = 15. The red dashed curve corresponds to Corollary 3. configuration of α, β, ν and z ∈ (0, 1) is not available in the literature, we present it in the following proposition. Proposition 1: Let A ∈ C m×m be Hermitian positive semi-definite with unit rank and tr(A) = a > 0. Then, for α, β, ν = 0, 1, 2, . . ., and z ∈ (0, 1), we have Z Im 0 detβ [Y ]detα [Im − zY ] … view at source ↗

**Figure 3.** The effect of p on the c.d.f. of scaled random variable λmax/p corresponding to the configuration m = n with ω = O(1) and ω = O(p). to arrive at the corresponding c.d.f. of λmax. In particular, The c.d.f. of the leading eigenvalue of singular F, for rank-one non-centrality parameter, is obtained from Fλmax (t; ω) in Theorem 3 by relabelling the parameters as follows m → p, p → m, and n → n + p − m. (30) Having armed… view at source ↗

Figures from the paper (5 more)

**Figure 4.** Figure 4: Limiting distributions of the scaled maximum eigenv [PITH_FULL_IMAGE:figures/full_fig_p020_4.png]

**Figure 5.** Figure 5: The effect of n, p and ω on ROC profile for m = 4. 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 [PITH_FULL_IMAGE:figures/full_fig_p024_5.png]

**Figure 6.** Figure 6: The behavior of ROC profile corresponding to the config [PITH_FULL_IMAGE:figures/full_fig_p024_6.png]

**Figure 7.** Figure 7: The effect of p on ROC profiles for m = n = 4 with ω = O(p) and ω = O(p 2 ). The blue curve depicts the limiting ROC profile given by 1 − (1 − PF ) 5/4 . B. High Dimensional Analysis Here we focus on the asymptotic characterization of the ROC of the leading eigenvalue …

**Figure 8.** Figure 8: High dimensional characteristics of the centered an [PITH_FULL_IMAGE:figures/full_fig_p029_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

78 extracted references · 78 canonical work pages

[12]

Couillet and M

R. Couillet and M. Debbah, Random Matrix Methods for Wireless Communications . Cambridge, U.K.: Cambridge University Press, 2011

work page 2011
[56]

On the moments of traces of two matrices in three situations for complex multivariate normal populati ons,

C. G. Khatri, “On the moments of traces of two matrices in three situations for complex multivariate normal populati ons,” Sankhy¯ a, vol. 32, no. 1, pp. 65–80, Mar. 1970

work page 1970
[1]

Detection of signals in colored noise: Leading eig envalue test for non-central f-matrices,

P . Dharmawansa, S. Atapattu, J. Evans, and K. Sithampara nathan, “Detection of signals in colored noise: Leading eig envalue test for non-central f-matrices,” in Proc. IEEE International Symposium on Information Theory , Athens, Greece, July 2024, pp. 539–544

work page 2024
[2]

Fundamental lim it of sample generalized eigenvalue based detection of sign als in noise using relatively few signal-bearing and noise-onl y samples,

R. R. Nadakuditi and J. W. Silverstein, “Fundamental lim it of sample generalized eigenvalue based detection of sign als in noise using relatively few signal-bearing and noise-onl y samples,” IEEE J. Sel. Top. Signal Process. , vol. 4, no. 3, pp. 468–480, Jun. 2010

work page 2010
[3]

Distribut ion of the scaled condition number of single-spiked complex Wishart matrices,

P . Dissanayake, P . Dharmawansa, and Y . Chen, “Distribut ion of the scaled condition number of single-spiked complex Wishart matrices,” IEEE Trans. Inf. Theory , vol. 68, no. 10, pp. 6716–6737, 2022

work page 2022
[4]

Eigenvalue-based detection of a signal in colored no ise: Finite and asymptotic analyses,

L. D. Chamain, P . Dharmawansa, S. Atapattu, and C. Tellam bura, “Eigenvalue-based detection of a signal in colored no ise: Finite and asymptotic analyses,” IEEE Trans. Inf. Theory , vol. 66, no. 10, pp. 6413–6433, 2020

work page 2020
[5]

Performan ce of statistical tests for single-source detection using r andom matrix theory,

P . Bianchi, M. Debbah, M. Maida, and J. Najim, “Performan ce of statistical tests for single-source detection using r andom matrix theory,” IEEE Trans. Inf. Theory , vol. 57, no. 4, pp. 2400–2419, Apr. 2011

work page 2011
[6]

Sample eigenvalue base d detection of high-dimensional signals in white noise usin g relatively few samples,

R. R. Nadakuditi and A. Edelman, “Sample eigenvalue base d detection of high-dimensional signals in white noise usin g relatively few samples,” IEEE Trans. Signal Process. , vol. 56, no. 7, pp. 2625 – 2638, 2008. DRAFT December 10, 2024 39

work page 2008

Show all 78 references

[7]

Passiv e multi-channel detection: A general ﬁrst-order statistic al theory,

T. McWhorter, L. Scharf, C. Moore, and M. Cheney, “Passiv e multi-channel detection: A general ﬁrst-order statistic al theory,” IEEE Open Journal of Signal Processing , vol. 43, pp. 437–451, 2023

2023
[8]

Testing in high-dimensi onal spiked models,

I. M. Johnstone and A. Onatski, “Testing in high-dimensi onal spiked models,” Ann. Stat. , vol. 44, no. 3, pp. 1231–1254, 2020

2020
[9]

Eigenvalue based spectrum sensi ng algorithms for cognitive radio,

Y . Zeng and Y . C. Liang, “Eigenvalue based spectrum sensi ng algorithms for cognitive radio,” IEEE Trans. Commun. , vol. 57, no. 6, pp. 3930 – 3941, 2009

2009
[10]

Spectru m sensing for cognitive radio: State-of-the-art and recent advances,

E. Axell, G. Leus, E. G. Larsson, and H. V . Poor, “Spectru m sensing for cognitive radio: State-of-the-art and recent advances,” IEEE Signal Process. Mag. , vol. 29, no. 3, pp. 101–116, May 2012

2012
[11]

MIM O radar moving target detection in homogeneous clutter,

Q. He, N. H. Lehmann, R. S. Blum, and A. M. Haimovich, “MIM O radar moving target detection in homogeneous clutter,” IEEE Trans. Aerosp. Electron. Syst. , vol. 46, no. 3, pp. 1290–1301, 2010

2010
[13]

On the distribution of the largest eig envalue in principal components analysis,

I. M. Johnstone, “On the distribution of the largest eig envalue in principal components analysis,” Ann. Statist. , vol. 29, no. 2, pp. 295–327, 2001

2001
[14]

K. V . Mardia, J. T. Kent, and C. C. Taylor, Multivariate Analysis . John Wiley & Sons, 2024, vol. 88

2024
[15]

Non-parametric detection of the number of signals: Hypothesis testing and random matr ix theory,

S. Kritchman and B. Nadler, “Non-parametric detection of the number of signals: Hypothesis testing and random matr ix theory,” IEEE Trans. Signal Process. , vol. 57, no. 9, pp. 3930 – 3941, 2009

2009
[16]

Roy’s largest root test u nder rank-one alternatives,

I. M. Johnstone and B. Nadler, “Roy’s largest root test u nder rank-one alternatives,” Biometrika, vol. 104, no. 1, pp. 181–193, Mar. 2017

2017
[17]

Roy’s large st root under rank-one perturbations: The complex valued ca se and applications,

P . Dharmawansa, B. Nadler, and O. Shwartz, “Roy’s large st root under rank-one perturbations: The complex valued ca se and applications,” J. Multivariate Anal. , vol. 174, p. 104524, Nov. 2019

2019
[18]

Extreme eigenvalues of large-dimen sional spiked Fisher matrices with application,

Q. Wang and J. Yao, “Extreme eigenvalues of large-dimen sional spiked Fisher matrices with application,” Ann. Statist. , vol. 45, no. 1, pp. 415–460, Feb. 2017

2017
[19]

Phase transition of t he largest eigenvalue for nonnull complex sample covarianc e matrices,

J. Baik, G. B. Arous, and S. Péché, “Phase transition of t he largest eigenvalue for nonnull complex sample covarianc e matrices,” Ann. Probab., vol. 33, no. 5, pp. 1643–1697, Sep. 2005

2005
[20]

Local asymptotic normality of the spectrum of high-dimensional s piked F-ratios,

P . Dharmawansa, I. M. Johnstone, and A. Onatski, “Local asymptotic normality of the spectrum of high-dimensional s piked F-ratios,” arXiv:1411.3875 [math.ST] , Nov. 2014

2014 arXiv
[21]

Mean-squared error and threshold SNR p rediction of maximum-likelihood signal parameter estimat ion with estimated colored noise covariances,

C. D. Richmond, “Mean-squared error and threshold SNR p rediction of maximum-likelihood signal parameter estimat ion with estimated colored noise covariances,” IEEE Trans. Inf. Theory , vol. 52, no. 5, pp. 2146–2164, 2006

2006
[22]

Statistic al inference in large antenna arrays under unknown noise pat tern,

J. Vinogradova, R. Couillet, and W. Hachem, “Statistic al inference in large antenna arrays under unknown noise pat tern,” IEEE Trans. Signal Process. , vol. 61, no. 22, pp. 5633–5645, Nov. 2013

2013
[23]

DOA estimation and detection in colored noise using additional noise-only data,

K. Werner and M. Jansson, “DOA estimation and detection in colored noise using additional noise-only data,” IEEE Trans. Signal Process. , vol. 55, no. 11, pp. 5309–5322, 2007

2007
[24]

Optimal utilization of signal-free samples for ar ray processing in unknown colored noise ﬁelds,

——, “Optimal utilization of signal-free samples for ar ray processing in unknown colored noise ﬁelds,” IEEE Trans. Signal Process., vol. 54, no. 10, pp. 3861–3872, 2006

2006
[25]

Generalized multivariat e analysis of variance–A uniﬁed framework for signal proces sing in correlated noise,

A. Dogandzic and A. Nehorai, “Generalized multivariat e analysis of variance–A uniﬁed framework for signal proces sing in correlated noise,” IEEE Signal Process. Mag. , vol. 20, no. 5, pp. 39–54, 2003

2003
[26]

Bayesian detecti on for MIMO radar in Gaussian clutter,

J. Liu, J. Han, Z.-J. Zhang, and J. Li, “Bayesian detecti on for MIMO radar in Gaussian clutter,” IEEE Trans. Signal Process., vol. 66, no. 24, pp. 6549–6559, 2018

2018
[27]

Adaptive rada r detection in Gaussian interference using clutter-free tr aining data,

Y . Rong, A. Aubry, A. De Maio, and M. Tang, “Adaptive rada r detection in Gaussian interference using clutter-free tr aining data,” IEEE Trans. Signal Process. , vol. 70, pp. 978–993, 2022. December 10, 2024 DRAFT 40

2022
[28]

Utilizati on of noise-only samples in array processing with prior know ledge,

D. Zachariah, M. Jansson, and M. Bengtsson, “Utilizati on of noise-only samples in array processing with prior know ledge,” IEEE Trans. Signal Process. Lett. , vol. 20, no. 9, pp. 865–868, 2013

2013
[29]

A STAP overview,

W. L. Melvin, “A STAP overview,” IEEE Aerospace and Electronic Sys. Mag. , vol. 19, no. 1, pp. 19–35, 2004

2004
[30]

Polyphase waveform des ign for MIMO radar space time adaptive processing,

B. Tang, J. Tuck, and P . Stoica, “Polyphase waveform des ign for MIMO radar space time adaptive processing,” IEEE Trans. Signal Process. , vol. 68, pp. 2143–2154, 2020

2020
[31]

MIMO radar waveform de sign with constant modulus and similarity constraints,

G. Cui, H. Li, and M. Rangaswamy, “MIMO radar waveform de sign with constant modulus and similarity constraints,” IEEE Trans. Signal Process. , vol. 62, no. 2, pp. 343–353, 2013

2013
[32]

MIMO radar with colocated antennas ,

J. Li and P . Stoica, “MIMO radar with colocated antennas ,” IEEE Signal Process. Mag. , vol. 24, no. 5, pp. 106–114, 2007

2007
[33]

MIMO radar with widely separated antennas,

A. M. Haimovich, R. S. Blum, and L. J. Cimini, “MIMO radar with widely separated antennas,” IEEE Trans. Aerosp. Electron. Syst. , vol. 25, no. 1, pp. 116–129, 2007

2007
[34]

Spatial diversity in radars—mo dels and detection performance,

E. Fishler, A. Haimovich, R. S. Blum, L. J. Cimini, D. Chi zhik, and R. A. V alenzuela, “Spatial diversity in radars—mo dels and detection performance,” IEEE Trans. Signal Process. , vol. 54, no. 3, pp. 823–838, 2006

2006
[35]

Adaptive rada r detection in Gaussian interference using clutter-free tr aining data,

Y . Rong, A. Aubry, A. De Maio, and M. Tang, “Adaptive rada r detection in Gaussian interference using clutter-free tr aining data,” IEEE Trans. Signal Process. , vol. 70, pp. 978–993, 2022

2022
[36]

Adaptive detection of Rician targets,

O. Besson, “Adaptive detection of Rician targets,” IEEE Trans. Aerosp. Electron. Syst. , vol. 59, no. 4, pp. 4700–4708, 2023

2023
[37]

Distributions of matrix variates and late nt roots derived from normal samples,

A. T. James, “Distributions of matrix variates and late nt roots derived from normal samples,” Ann. Math. Statist. , vol. 35, no. 2, pp. 475–501, Jun. 1964

1964
[38]

R. J. Muirhead, Aspects of Multivariate Statistical Theory . John Wiley & Sons, 2009, vol. 197

2009
[39]

On the performance of MUSIC w ith Toeplitz rectiﬁcation in the context of large arrays,

P . V allet and P . Loubaton, “On the performance of MUSIC w ith Toeplitz rectiﬁcation in the context of large arrays,” IEEE Trans. Signal Process. , vol. 65, no. 22, pp. 5848–5859, 2017

2017
[40]

Performance an alysis of spatial smoothing schemes in the context of large a rrays,

G.-T. Pham, P . Loubaton, and P . V allet, “Performance an alysis of spatial smoothing schemes in the context of large a rrays,” IEEE Trans. Signal Process. , vol. 64, no. 1, pp. 160–172, 2015

2015
[41]

Modiﬁed subspace algorith ms for DoA estimation with large arrays,

X. Mestre and M. Á. Lagunas, “Modiﬁed subspace algorith ms for DoA estimation with large arrays,” IEEE Trans. Signal Process., vol. 56, no. 2, pp. 598–614, 2008

2008
[42]

Multivariate analysis and Jacobi ens embles: Largest eigenvalue, Tracy-Widom limits and rates o f convergence,

I. M. Johnstone, “Multivariate analysis and Jacobi ens embles: Largest eigenvalue, Tracy-Widom limits and rates o f convergence,” Ann. Stat. , vol. 36, no. 6, pp. 2638–2716, 2008

2008
[43]

Invariance principl e and CLT for the spiked eigenvalues of large-dimensional Fi sher matrices and applications,

D. Jiang, Z. Hou, Z. Bai, and R. Li, “Invariance principl e and CLT for the spiked eigenvalues of large-dimensional Fi sher matrices and applications,” arXiv preprint arXiv:2203.14248 , 2022

2022 arXiv
[44]

The Tracy–Widom law for the largest eigenvalue of F type matrices,

X. Han, G. Pan, and B. Zhang, “The Tracy–Widom law for the largest eigenvalue of F type matrices,” Ann. Stat. , vol. 44, no. 4, pp. 1564–1592, 2016

2016
[45]

A uniﬁed matrix model includ ing both CCA and F-matrices in multivariate analysis: The largest eigenvalue and its applications,

X. Han, G. Pan, and Q. Yang, “A uniﬁed matrix model includ ing both CCA and F-matrices in multivariate analysis: The largest eigenvalue and its applications,” Bernoulli, vol. 24, no. 4B, pp. 3447–3468, 2018

2018
[46]

The limits of the sample spik ed eigenvalues for a high-dimensional generalized Fisher m atrix and its applications,

D. Jiang, Z. Hou, and J. Hu, “The limits of the sample spik ed eigenvalues for a high-dimensional generalized Fisher m atrix and its applications,” Journal of Statistical Planning and Inference , vol. 215, pp. 208–217, 2021

2021
[47]

Extreme eigenvalues of large-dimen sional spiked Fisher matrices with application,

Q. Wang and J. Yao, “Extreme eigenvalues of large-dimen sional spiked Fisher matrices with application,” Ann. Stat. , vol. 15, no. 1, pp. 415–460, 2017

2017
[48]

Spiked eigenvalues o f non-central Fisher matrix with applications,

Z. Hou, X. Zhang, Z. Bai, and J. Hu, “Spiked eigenvalues o f non-central Fisher matrix with applications,” Bernoulli, vol. 29, no. 4, pp. 3171–3197, 2023

2023
[49]

Phase transition of t he largest eigenvalue for nonnull complex sample covarianc e matrices,

J. Baik, G. B. Arous, and S. Péché, “Phase transition of t he largest eigenvalue for nonnull complex sample covarianc e matrices,” Ann. Probab., vol. 33, no. 5, pp. 1643–1697, Sep. 2005. DRAFT December 10, 2024 41

2005
[50]

MUSIC, G-MUS IC, and maximum-likelihood performance breakdown,

B. Johnson, Y . Abramovich, and X. Mestre, “MUSIC, G-MUS IC, and maximum-likelihood performance breakdown,” IEEE Trans. Signal Process. , vol. 56, no. 8, pp. 3944–3958, Aug. 2008

2008
[51]

The probability of a subspace swap in the SVD,

J. Thomas, L. Scharf, and D. Tufts, “The probability of a subspace swap in the SVD,” IEEE Trans. Signal Process. , vol. 43, no. 3, pp. 730–736, Mar. 1995

1995
[52]

The threshold analysi s of SVD-based algorithms,

D. Tufts, A. Kot, and R. V accaro, “The threshold analysi s of SVD-based algorithms,” in Proc. IEEE ICASSP , New Y ork, NY , USA, Apr. 1988, pp. 2416–2419

1988
[53]

M. L. Mehta, Random Matrices . Academic Press, 2004, vol. 142

2004
[54]

Improved detection o f correlated signals in low-rank-plus-noise type data sets using informative canonical correlation analysis (ICCA),

N. Asendorf and R. R. Nadakuditi, “Improved detection o f correlated signals in low-rank-plus-noise type data sets using informative canonical correlation analysis (ICCA),” IEEE Trans. Inf. Theory , vol. 63, no. 6, pp. 3451–3467, 2017

2017
[55]

Zonal Polynomials,

A. Takemura, “Zonal Polynomials,” Lecture Notes-Monograph Series , vol. 4, pp. 1–104, 1984

1984
[57]

Total positivity, sph erical series, and hypergeometric functions of matrix argu ment,

K. I. Gross and D. S. P . Richards, “Total positivity, sph erical series, and hypergeometric functions of matrix argu ment,” Journal of Approximation theory , vol. 59, no. 2, pp. 224–246, 1989

1989
[58]

Gradshteyn and I

I. Gradshteyn and I. Ryzhik, Table of Integrals, Series, and Products , 7th ed. Boston: Academic Press, 2007

2007
[59]

Bateman and A

H. Bateman and A. Erdélyi, Higher Transcendental Functions: vol. 1 , ser. California Institute of Technology. Bateman Manuscript Project. New Y ork, NY: McGraw-Hill, 1955

1955
[60]

The rank-1 real Wishart spiked model,

M. Y . Mo, “The rank-1 real Wishart spiked model,” Commun. Pure Appl. Math. , no. 65, pp. 1528–1638, Nov. 2012

2012
[61]

Probability densities and distribut ions for spiked and general variance Wishart β -ensembles,

P . J. Forrester, “Probability densities and distribut ions for spiked and general variance Wishart β -ensembles,” Random Matrices: Theory and Applications , vol. 2, no. 4, 2013

2013
[62]

A. M. Mathai, Jacobians of Matrix Transformation and Functions of Matrix Arguments. World Scientiﬁc Publishing Company, 1997

1997
[63]

Densities of the extreme eigenvalue s of Beta–MANOV A matrices,

R. Kan and P . Koev, “Densities of the extreme eigenvalue s of Beta–MANOV A matrices,” Random Matrices: Theory Appl. , vol. 8, no. 01, p. 1950002, 2019

2019
[64]

Smallest eigenvalue distributions for t wo classes of β -Jacobi ensembles,

I. Dumitriu, “Smallest eigenvalue distributions for t wo classes of β -Jacobi ensembles,” Journal of Mathematical Physics , vol. 53, no. 10, p. 103301, 2012

2012
[65]

Distribution of the extreme ei genvalues of the complex Jacobi random matrix ensemble,

P . Koev and I. Dumitriu, “Distribution of the extreme ei genvalues of the complex Jacobi random matrix ensemble,” SIAM. J. Matrix Anal. & Appl. , 2005

2005
[66]

On singular wishart and singular multivaria te beta distributions,

H. Uhlig, “On singular wishart and singular multivaria te beta distributions,” The Annals of Statistics , pp. 395–405, 1994

1994
[67]

Some results for the singular normal mult ivariate regression models,

C. G. Khatri, “Some results for the singular normal mult ivariate regression models,” Sankhy¯ a: The Indian Journal of Statistics, Series A , pp. 267–280, 1968

1968
[68]

Complex singular W ishart matrices and applications,

T. Ratnarajah and R. V aillancourt, “Complex singular W ishart matrices and applications,” Computers & Mathematics with Applications, vol. 50, no. 3-4, pp. 399–411, 2005

2005
[69]

Singular matrix var iate beta distribution,

J. A. Díaz-García and R. G. Jáimez, “Singular matrix var iate beta distribution,” Journal of Multivariate Analysis , vol. 99, no. 4, pp. 637–648, 2008

2008
[70]

On the singular gamma, Wi shart, and beta matrix-variate density functions,

A. M. Mathai and S. B. Provost, “On the singular gamma, Wi shart, and beta matrix-variate density functions,” Canadian Journal of Statistics , vol. 50, no. 4, pp. 1143–1165, 2022

2022
[71]

R. A. Horn and C. R. Johnson, Matrix Analysis . Cambridge university press, 2012

2012
[72]

A. W. Marshall, I. Olkin, and B. C. Arnold, Inequalities: Theory of Majorization and Its Applications . Springer, 1979

1979
[73]

Level-spacing distributions and the Airy kernel,

C. A. Tracy and H. Widom, “Level-spacing distributions and the Airy kernel,” Communications in Mathematical Physics , vol. 159, pp. 151–174, 1994

1994
[74]

P . J. Forrester, Log-Gases and Random Matrices (LMS-34) . Princeton, NJ: Princeton University Press, 2010. December 10, 2024 DRAFT 42

2010
[75]

The strong limits of random matrix spect ra for sample matrices of independent elements,

K. W. Wachter, “The strong limits of random matrix spect ra for sample matrices of independent elements,” Ann. Probab., vol. 6, no. 1, pp. 1–18, Feb. 1978

1978
[76]

The limiting eigenvalue distributio n of a multivariate F-matrix,

J. Silverstein, “The limiting eigenvalue distributio n of a multivariate F-matrix,” SIAM J. Math. Anal. , vol. 16, no. 3, pp. 641–646, 1985

1985
[77]

A. M. Mathai, S. B. Provost, and T. Hayakawa, Bilinear F orms and Zonal Polynomials . Springer Science & Business Media, 1995, vol. 102

1995
[78]

On the capacity of spa tially correlated MIMO Rayleigh-fading channels,

M. Chiani, M. Win, and A. Zanella, “On the capacity of spa tially correlated MIMO Rayleigh-fading channels,” IEEE Trans. Inf. Theory , vol. 49, no. 10, pp. 2363–2371, Oct. 2003. DRAFT December 10, 2024

2003

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [12]

Couillet and M

R. Couillet and M. Debbah, Random Matrix Methods for Wireless Communications . Cambridge, U.K.: Cambridge University Press, 2011

work page 2011

[2] [56]

On the moments of traces of two matrices in three situations for complex multivariate normal populati ons,

C. G. Khatri, “On the moments of traces of two matrices in three situations for complex multivariate normal populati ons,” Sankhy¯ a, vol. 32, no. 1, pp. 65–80, Mar. 1970

work page 1970

[3] [1]

Detection of signals in colored noise: Leading eig envalue test for non-central f-matrices,

P . Dharmawansa, S. Atapattu, J. Evans, and K. Sithampara nathan, “Detection of signals in colored noise: Leading eig envalue test for non-central f-matrices,” in Proc. IEEE International Symposium on Information Theory , Athens, Greece, July 2024, pp. 539–544

work page 2024

[4] [2]

Fundamental lim it of sample generalized eigenvalue based detection of sign als in noise using relatively few signal-bearing and noise-onl y samples,

R. R. Nadakuditi and J. W. Silverstein, “Fundamental lim it of sample generalized eigenvalue based detection of sign als in noise using relatively few signal-bearing and noise-onl y samples,” IEEE J. Sel. Top. Signal Process. , vol. 4, no. 3, pp. 468–480, Jun. 2010

work page 2010

[5] [3]

Distribut ion of the scaled condition number of single-spiked complex Wishart matrices,

P . Dissanayake, P . Dharmawansa, and Y . Chen, “Distribut ion of the scaled condition number of single-spiked complex Wishart matrices,” IEEE Trans. Inf. Theory , vol. 68, no. 10, pp. 6716–6737, 2022

work page 2022

[6] [4]

Eigenvalue-based detection of a signal in colored no ise: Finite and asymptotic analyses,

L. D. Chamain, P . Dharmawansa, S. Atapattu, and C. Tellam bura, “Eigenvalue-based detection of a signal in colored no ise: Finite and asymptotic analyses,” IEEE Trans. Inf. Theory , vol. 66, no. 10, pp. 6413–6433, 2020

work page 2020

[7] [5]

Performan ce of statistical tests for single-source detection using r andom matrix theory,

P . Bianchi, M. Debbah, M. Maida, and J. Najim, “Performan ce of statistical tests for single-source detection using r andom matrix theory,” IEEE Trans. Inf. Theory , vol. 57, no. 4, pp. 2400–2419, Apr. 2011

work page 2011

[8] [6]

Sample eigenvalue base d detection of high-dimensional signals in white noise usin g relatively few samples,

R. R. Nadakuditi and A. Edelman, “Sample eigenvalue base d detection of high-dimensional signals in white noise usin g relatively few samples,” IEEE Trans. Signal Process. , vol. 56, no. 7, pp. 2625 – 2638, 2008. DRAFT December 10, 2024 39

work page 2008