Pith. sign in

REVIEW 3 major objections 3 minor 37 references

This paper proves that the nonconvex maximum-likelihood objective for Toeplitz covariance estimation, parameterized by Carathéodory amplitudes on a fixed frequency grid, has no spurious stationary points—every stationary point with a positi

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 00:18 UTC pith:4JPBKZTI

load-bearing objection A useful overparameterized-GD algorithm, but the main theoretical result is false as stated—missing model-inclusion assumption. the 3 major comments →

arxiv 2511.01605 v2 pith:4JPBKZTI submitted 2025-11-03 cs.LG stat.ML

The Optimization Landscape of Carath\'eodory Decomposition of Toeplitz Covariances

Daniel Busbib , Ami Wiesel This is my paper
classification cs.LG stat.ML MSC 62F1090C2662H12
keywords Toeplitz covariance estimationCarathéodory decompositionnonconvex optimizationgradient descentoverparameterizationmaximum likelihood estimationstationary pointsspectral estimation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper studies recovering a Toeplitz covariance matrix from Gaussian samples via a Carathéodory decomposition—writing the covariance as a sum of complex sinusoids with learnable amplitudes and frequencies. It aims to establish when this nonconvex maximum-likelihood formulation is benign enough that simple gradient descent reaches the global optimum. The central theorem proves that when the frequencies are fixed and the true covariance lies on that grid, every stationary point of the amplitude-only negative log-likelihood recovers the true covariance exactly in the population limit, and a companion result bounds the error under small sample perturbation. The paper also reports that jointly optimizing amplitudes and frequencies with mild overparameterization (K=2P or 4P) lets plain gradient descent approach the Cramér-Rao bound, whereas the minimally parameterized K=P case often fails, and that fixed-grid amplitude-only methods have a positive error floor when true frequencies are off-grid. If right, the work explains why a nonconvex problem that might 'scare researchers' can in fact be solved by a simple algorithm, and it points toward a principled use of overparameterization in structured covariance estimation.

Core claim

The paper's central claim is Theorem 1: in the population setting (sample covariance equals the true covariance S=C), if the sinusoids {v(ω_k)} span R^P and bC(â) is positive definite, then any stationary point â of the amplitude-only negative log-likelihood NLL(a) satisfies bC(â)=C. In other words, the nonconvex objective has no spurious local minima—every stationary point is globally optimal. Theorem 2 extends this to the finite-sample setting: when S=C+Δ with ∥Δ∥_F≤ε, any stationary point with a well-conditioned estimate satisfies ∥bC(â)−C∥_F^2 ≤ (λ^2/μ^2) ε^2. The paper also claims, based on experiments, that overparameterized joint optimization (amplitudes and frequencies) with K≈2P or

What carries the argument

The key object is the Carathéodory decomposition C = Σ a_k v(ω_k)v(ω_k)^H + εI, which guarantees positive-definiteness and lets the optimization run unconstrained. The proof of the benign landscape uses the identity that the stationarity condition v_k^H M(â) v_k = 0, with M = bC^{-1}(bC−C)bC^{-1}, is exactly the statement that the error Γ = bC(â)−C is orthogonal to every rank-one steering term under the weighted inner product ⟨X,Y⟩_{bC} = tr(bC^{-1} X bC^{-1} Y). Since the steering vectors span R^P, the error matrix belongs to the span of these rank-one terms, so Γ must be orthogonal to itself and therefore zero. The same inner-product machinery converts the perturbation analysis into a Cauc

Load-bearing premise

The benign-landscape theorem assumes the true covariance can be written exactly as a sum of sinusoids on the fixed frequency grid; if the true frequencies are off-grid, the model cannot represent the truth and the theorem's conclusion does not apply.

What would settle it

Run the proposed amplitude-only gradient algorithm on a true Toeplitz covariance that is exactly representable on a fixed grid, using many random initializations; if any run converges to a positive-definite stationary covariance that differs noticeably from the true one, Theorem 1 is false. A simpler tabular test: construct a small P and K=P grid, enumerate stationary points numerically, and check whether any yields bC ≠ C; the proof predicts none do.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • For the amplitude-only subproblem on a fixed grid, any local search method—gradient descent, coordinate descent, or otherwise—cannot get stuck at a spurious stationary point in the population limit; it will converge to the true covariance.
  • The perturbation bound implies the nonconvex maximum-likelihood estimator is stable at finite sample sizes: as the sample covariance approaches the true one, every stationary point approaches the truth, so consistent estimation does not require convexifying the likelihood.
  • The empirical success at K=2P and K=4P suggests mild overparameterization is enough to make gradient descent globally convergent for the joint problem, even though the theory for joint optimization is not yet proven.
  • The separate-learning-rate scheme, motivated by the derived Lipschitz constants for amplitude vs frequency blocks, provides a practical 3–5× convergence speedup and indicates block-wise preconditioning is a useful tool for this problem.
  • The grid-mismatch result warns that amplitude-only methods on a fixed frequency grid, such as some classical spectral-estimation approaches, cannot reach zero error when the true frequencies are off the grid, no matter how many grid points are used.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If the benign-landscape theorem can be extended from the amplitude-only subproblem to the joint (amplitude + frequency) objective, it would fully justify using simple gradient descent on the Carathéodory parameterization, making the MLE computationally trivial at scale. This is a natural next step, since the paper states joint analysis is beyond its scope.
  • The closed-form Hessian and curvature approximation suggest second-order or quasi-Newton methods could be designed to exploit the structure, potentially matching ATOM's accuracy with far lower per-iteration cost.
  • The finding that K=P often fails but K=2P succeeds mirrors the overparameterization phenomenon in neural network training; a formal convergence-rate bound as a function of K, P, and sample size would connect this work to the broader theory of overparameterized optimization.
  • The requirement that the true covariance be exactly on the fixed frequency grid is strong; in practice one would want a version of the theorem for model mismatch, showing that stationary points are near the best approximation of C on the grid rather than necessarily equal to it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The paper proposes estimating Toeplitz covariance matrices by maximizing the Gaussian likelihood over an overparameterized Carathéodory decomposition, i.e., a sum of complex sinusoids with learnable amplitudes and frequencies. The authors derive closed-form gradients and Hessian blocks, propose a GD variant with separate learning rates for amplitudes and frequencies, and report experiments showing that mild overparameterization (K=2P or 4P) helps GD reach near-CRB accuracy while K=P often fails. The theoretical contribution is an analysis of the amplitude-only optimization landscape when frequencies are fixed. Theorem 1 claims that, in the population setting S=C, every stationary point with positive definite ĤC(Ĥa) recovers the true covariance; Theorem 2 extends this to a perturbation bound when S=C+Δ. The full-text abstract limits the landscape claim to fixed frequencies, and Section VII states that joint amplitude-frequency optimization is not proved.

Significance. The empirical part of this paper is potentially useful: the algorithm is simple, reproducible (open-source code is provided), and the numerical experiments show that overparameterized GD with separate learning rates is competitive with or better than ATOM and PGD in several settings. The derivation of gradients and the Hessian-based motivation for separate step sizes are also nice practical contributions. The theoretical landscape result, if correct, would be interesting because the amplitude-only NLL is nonconvex, and a no-spurious-stationary-point theorem would justify direct minimization. However, the stated Theorem 1 is false without an additional model-inclusion assumption, and Theorem 2 inherits the same problem. The proof technique itself is clean and correct under the missing assumption, so the paper can be repaired, but the current version's central theoretical claim is not valid as stated. The mismatch between the arXiv abstract's joint-optimization claim and the manuscript's actual fixed-frequency theorem is another load-bearing inconsistency.

major comments (3)
  1. [§V, Theorem 1 / Appendix A, Eq. (29)] Theorem 1 is false as stated. The hypothesis that {v(ω_k)} spans R^P gives full row rank of V but does not imply the true Toeplitz C lies in the fixed-grid model V diag(a)V^H; the proof silently assumes this in Eq. (29). Counterexample: P=2, K=2, ω=(0,π), V=[[1,1],[1,-1]], C=[[1,j/2],[-j/2,1]]. At Ĥa=(1/2,1/2), ĤC=V diag(Ĥa)V^H=I≻0 and direct differentiation gives ∇NLL=0, but ĤC≠C. Hence the theorem's conclusion fails under its stated assumptions. It can be repaired by adding the assumption C∈span{v_k v_k^H}; the proof is then correct. Also, the proof's ĤC=Vdiag(Ĥa)V^H omits the softplus and εI of the model in (5), so the theorem must specify the exact parameterization.
  2. [Theorem 2 / Appendix B, Eq. (49) and following] Theorem 2 inherits the same hidden model-inclusion assumption. The proof states 'Since ĤC,C∈S (both are feasible Toeplitz covariances)', but C being a positive definite Toeplitz matrix does not imply C∈S=span{v_k v_k^H}. The assumption Γ=ĤC−C∈S is needed for the argument at Eqs. (54)–(55). Without adding C∈S to the theorem statement, the bound (26) is unsupported. Please add this assumption explicitly.
  3. [Abstract and Section VII] The arXiv abstract claims the main theoretical result is that joint amplitude-frequency optimization has a benign population landscape ('every stationary point ... recovers the true Toeplitz covariance'). No such theorem is proved in the manuscript; Section VII explicitly states 'While we cannot yet prove this for joint amplitude and frequency optimization'. The full-text abstract correctly limits the landscape claim to fixed frequencies. This mismatch is load-bearing because it changes the paper's central claim and must be corrected in the published version.
minor comments (3)
  1. [§V and Appendix A] The notation ĤC(Ĥa) switches between the softplus/εI model of Eq. (5) and the plain Vdiag(Ĥa)V^H model of Eq. (29). The theorems should state explicitly which parameterization and which objective they analyze.
  2. [Theorems 1–2] The phrase 'span R^P' should be 'span C^P' (or 'V has full row rank'), since the steering vectors v(ω_k) are complex.
  3. [Table I] The numerical entries in Table I are difficult to read (e.g., '101.2870 ×8.6' and '101.11012 ×10.0'). Please use standard scientific notation or explicit powers of ten.

Circularity Check

0 steps flagged

No significant circularity: the proof is self-contained from stated stationary-point conditions; the main risk is an unstated model-inclusion assumption, which is a correctness gap rather than a circular reduction.

full rationale

The central theoretical claim, Theorem 1, is derived from the stationary condition v_k^H M(ba) v_k = 0 and a weighted inner product argument in Appendix A (Eqs. (33)–(47)); it is not produced by fitting a parameter to the target and renaming the fit. The population NLL is minimized at the true covariance by construction, but the theorem's content is that every stationary point with positive definite estimate equals the true covariance, which is a genuine mathematical statement. The fixed-frequency amplitude-only analysis does rely on Eq. (29), C = V diag(a) V^H, an assumption that true C lies in the fixed-grid model. This is a hidden model-inclusion condition not implied by the stated span hypothesis, and the abstract's grid-mismatch error-floor claim appears inconsistent with Theorem 1 as stated. That is a correctness concern, not circularity: the proof does not define C in terms of bC(ba), nor does the conclusion follow by definition from the assumption. Theorem 2 follows by the same inner-product argument with a perturbation bound, and no external result is imported. The self-citations, including the CAMSAP preliminary version [1], are incidental and not load-bearing: no prior uniqueness theorem or ansatz is invoked to force the choice of model or objective. The empirical overparameterized-GD results are benchmark comparisons against ATOM and PGD, not predictions derived from the theorems. Thus no circular step is exhibited.

Axiom & Free-Parameter Ledger

3 free parameters · 5 axioms · 0 invented entities

No new physical or mathematical entities are invented. The core proof depends on a hidden model-inclusion assumption (C∈S) plus standard Gaussian/Toeplitz assumptions, which is why soundness is capped.

free parameters (3)
  • K (number of sinusoids) = P, 2P, 4P, 50P in experiments
    Model complexity/overparameterization factor chosen by the user; the paper's main empirical claim is that K≈2P–4P gives global convergence while K=P fails.
  • ε (fixed regularization floor)
    In Eq. (5), εI ensures positive definiteness and is kept fixed; no value is reported, so it is an unspecified hyperparameter affecting whether Ĉ can equal C.
  • learning rates and line-search constants = α=0.3, β=0.5; initial η0, η0^(a), η0^(ω) not fully specified
    Algorithm hyperparameters needed to reproduce convergence claims; not fitted to data but freely chosen.
axioms (5)
  • standard math Carathéodory decomposition: every positive definite Toeplitz C can be expressed as Σ a_k v(ω_k)v(ω_k)^H, possibly plus σ²I.
    Used to justify the parameterization in Eq. (1).
  • domain assumption Gaussian zero-mean model and sample covariance sufficiency.
    Section II defines x∼CN(0,C), S=(1/M)Σxx^H; the negative log-likelihood in Eq. (8) relies on this.
  • ad hoc to paper True covariance lies in the fixed-grid model class: C=V diag(a)V^H.
    Introduced silently in Appendix A Eq. (29); not stated in Theorem 1, whose 'span R^P' condition is insufficient.
  • domain assumption Existence of a positive definite stationary point.
    Theorem 1's conclusion applies only if such a point exists; otherwise the statement is vacuous.
  • domain assumption Bounded conditioning μI ⪯ Ĉ ⪯ λI and ∥Δ∥_F ≤ ε in the perturbation setting.
    Theorem 2 requires these bounds; they restrict the regime of the stability result.

pith-pipeline@v1.3.0-alltime-deepseek · 13096 in / 19344 out tokens · 205457 ms · 2026-08-04T00:18:24.419095+00:00 · methodology

0 comments
read the original abstract

Toeplitz covariance estimation is a classical problem in statistical signal processing, yet the geometry of the Gaussian maximum-likelihood objective remains only partially understood. Recent algorithms, including Newton-type, majorization-minimization, and gradient-based methods, indicate that the nonconvex problem can often be globally solved when the number of samples is sufficiently large, but they also reveal a difficult computational landscape. In this work, we study this phenomenon through an overparameterized Caratheodory representation of positive definite Toeplitz covariance matrices. The Caratheodory decomposition parameterizes the covariance using a combination of steering vectors with different frequencies and amplitudes. Our first result shows that fixed-grid amplitude optimization is fundamentally insufficient. Even in the population setting, and even with arbitrarily many fixed frequency grid points, amplitude-only optimization can have a strictly positive error floor under grid mismatch. This motivates optimizing both amplitudes and frequencies. In this case, our main theoretical result proves that the joint optimization has a benign population landscape: every stationary point that produces a positive definite covariance matrix recovers the true Toeplitz covariance. These findings suggest a simple interpretation of the Toeplitz covariance problem: the population landscape is globally benign, but may be highly ill-conditioned. In our numerical experiments, overparameterization improves convergence speed and finite-sample accuracy. In particular, it allows simple gradient descent to approach the Cramer Rao bound while keeping the implementation simple.

Figures

Figures reproduced from arXiv: 2511.01605 by Ami Wiesel, Daniel Busbib.

Figure 1
Figure 1. Figure 1: Validation of empirical Lipschitz approximation across 1000 Monte Carlo trials. [PITH_FULL_IMAGE:figures/full_fig_p005_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: RMSE versus sample size for the ATOM benchmark [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: RMSE versus sample size for AR(3) covariance model. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: RMSE versus sample size M for different overpa￾rameterization factors K ∈ [1, 2]. The results indicate that overparameterized models (K ≈ 2P) achieve substantially lower RMSE, even for small M, whereas minimally parame￾terized configurations (K ≈ P) remain unstable and sensitive to sample size. domly generated cases that do not follow any particular pattern. Table II reports the runtime complexity (in Matl… view at source ↗
Figure 4
Figure 4. Figure 4: RMSE versus sample size for random Carath [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 6
Figure 6. Figure 6: Gradient descent computation time comparison be [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 3 linked inside Pith

  1. [1]

    Toeplitz covariance estimation via over- parametrized gradient descent,

    D. Busbib and A. Wiesel, “Toeplitz covariance estimation via over- parametrized gradient descent,” inProc. IEEE CAMSAP, 2025, accepted

    2025

  2. [2]

    Kay,Fundamentals of Statistical Signal Processing: Detection theory

    S. Kay,Fundamentals of Statistical Signal Processing: Detection theory. Prentice-Hall PTR, 1998

    1998

  3. [3]

    Structured robust covariance estimation,

    A. Wiesel, T. Zhanget al., “Structured robust covariance estimation,” Foundations and Trends® in Signal Processing, vol. 8, no. 3, pp. 127– 216, 2015

    2015

  4. [4]

    A geometric approach to covariance matrix estimation and its applications to radar problems,

    A. Aubry, A. De Maio, and L. Pallotta, “A geometric approach to covariance matrix estimation and its applications to radar problems,” IEEE Transactions on Signal Processing, vol. 66, no. 4, pp. 907–922, 2018

    2018

  5. [5]

    Application of Toeplitz covariance estimation to adaptive beamforming and detection,

    D. Fuhrmann, “Application of Toeplitz covariance estimation to adaptive beamforming and detection,”IEEE Transactions on Signal Processing, vol. 39, no. 10, pp. 2194–2198, 1991

    1991

  6. [6]

    Detection algorithms in hyperspectral imaging systems: An overview of practical algorithms,

    D. Manolakis, E. Truslow, M. Pieper, T. Cooley, and M. Brueggeman, “Detection algorithms in hyperspectral imaging systems: An overview of practical algorithms,”IEEE Signal Processing Magazine, vol. 31, no. 1, pp. 24–33, 2014

    2014

  7. [7]

    Computationally efficient maximum likeli- hood estimation of structured covariance matrices,

    H. Li, P. Stoica, and J. Li, “Computationally efficient maximum likeli- hood estimation of structured covariance matrices,”IEEE Transactions on Signal Processing, vol. 47, no. 5, pp. 1314–1323, 1999

    1999

  8. [8]

    On the asymptotic behavior of the sample estimates of eigenvalues and eigenvectors of covariance matrices,

    X. Mestre, “On the asymptotic behavior of the sample estimates of eigenvalues and eigenvectors of covariance matrices,”IEEE Transactions on Signal Processing, vol. 56, no. 11, pp. 5353–5368, 2008

    2008

  9. [9]

    Covariance, subspace, and intrinsic Cram ´er-Rao bounds,

    S. T. Smith, “Covariance, subspace, and intrinsic Cram ´er-Rao bounds,” IEEE Transactions on Signal Processing, vol. 53, no. 5, pp. 1610–1630, 2005

    2005

  10. [10]

    Sample efficient Toeplitz covariance estimation,

    Y . C. Eldar, J. Li, C. Musco, and C. Musco, “Sample efficient Toeplitz covariance estimation,” inProceedings of the Fourteenth Annual ACM- SIAM Symposium on Discrete Algorithms. SIAM, 2020, pp. 378–397

    2020

  11. [11]

    The role of likelihood and entropy in incomplete-data problems: Applications to estimating point-process in- tensities and Toeplitz constrained covariances,

    M. Miller and D. Snyder, “The role of likelihood and entropy in incomplete-data problems: Applications to estimating point-process in- tensities and Toeplitz constrained covariances,”Proceedings of the IEEE, vol. 75, no. 7, pp. 892–907, 1987

    1987

  12. [12]

    On the asymptotic equivalence of circulant and Toeplitz matrices,

    Z. Zhu and M. B. Wakin, “On the asymptotic equivalence of circulant and Toeplitz matrices,”IEEE Transactions on Information Theory, vol. 63, no. 5, pp. 2975–2992, 2017

    2017

  13. [13]

    Optimal rates of convergence for covariance matrix estimation,

    T. T. Cai, C.-H. Zhang, and H. H. Zhou, “Optimal rates of convergence for covariance matrix estimation,”The Annals of Statistics, vol. 38, no. 4, pp. 2118–2144, 2010

    2010

  14. [14]

    Maximum-likelihood estimation of com- plex sinusoids and Toeplitz covariances,

    M. Turmon and M. Miller, “Maximum-likelihood estimation of com- plex sinusoids and Toeplitz covariances,”IEEE Transactions on Signal Processing, vol. 42, no. 5, pp. 1074–1086, 1994

    1994

  15. [15]

    MELT—Maximum-Likelihood Estimation of Low-Rank Toeplitz Covariance Matrix,

    P. Babu, “MELT—Maximum-Likelihood Estimation of Low-Rank Toeplitz Covariance Matrix,”IEEE Signal Processing Letters, vol. 23, no. 11, pp. 1587–1591, 2016

    2016

  16. [16]

    Toeplitz structured covari- ance matrix estimation for radar applications,

    X. Du, A. Aubry, A. De Maio, and G. Cui, “Toeplitz structured covari- ance matrix estimation for radar applications,”IEEE Signal Processing Letters, vol. 27, pp. 595–599, 2020

    2020

  17. [17]

    Advanced methods for MLE of Toeplitz structured covariance matrices with applications to radar problems,

    A. Aubry, P. Babu, A. De Maio, and M. Rosamilia, “Advanced methods for MLE of Toeplitz structured covariance matrices with applications to radar problems,”IEEE Transactions on Information Theory, vol. 70, no. 12, pp. 9277–9292, 2024

    2024

  18. [18]

    Gohberg- Semencul Toeplitz covariance estimation via autoregressive parameters,

    B. B ¨ock, D. Semmler, B. Fesl, M. Baur, and W. Utschick, “Gohberg- Semencul Toeplitz covariance estimation via autoregressive parameters,” IEEE Transactions on Signal Processing, vol. 73, pp. 858–875, 2025

    2025

  19. [19]

    On the connection between the extremes of harmonic functions and their coefficients and on Picard–Landau’s theorem,

    C. Carath ´eodory and L. Fej ´er, “On the connection between the extremes of harmonic functions and their coefficients and on Picard–Landau’s theorem,”Rendiconti del Circolo Matematico di Palermo, vol. 32, no. 1, pp. 218–239, 1884

  20. [20]

    Grenander and G

    U. Grenander and G. Szeg ˝o,Toeplitz Forms and Their Applications, ser. California Monographs in Mathematical Sciences. Berkeley and Los Angeles: University of California Press, 1958

    1958

  21. [21]

    Stoica and R

    P. Stoica and R. L. Moses,Spectral Analysis of Signals. Upper Saddle River, NJ: Pearson Prentice Hall, 2005

    2005

  22. [22]

    High-resolution frequency-wavenumber spectrum analysis,

    J. Capon, “High-resolution frequency-wavenumber spectrum analysis,” Proceedings of the IEEE, vol. 57, no. 8, pp. 1408–1418, 1969

    1969

  23. [23]

    Multiple emitter location and signal parameter estimation,

    R. Schmidt, “Multiple emitter location and signal parameter estimation,” IEEE Transactions on Antennas and Propagation, vol. 34, no. 3, pp. 276–280, 1986

    1986

  24. [24]

    ESPRIT—estimation of signal parameters via rotational invariance techniques,

    R. Roy and T. Kailath, “ESPRIT—estimation of signal parameters via rotational invariance techniques,”IEEE Transactions on Acoustics, Speech and Signal Processing, vol. 37, pp. 984–995, 1989. 8

    1989

  25. [25]

    SPICE: A sparse covariance-based estimation method for array processing,

    P. Stoica, P. Babu, and J. Li, “SPICE: A sparse covariance-based estimation method for array processing,”IEEE Transactions on Signal Processing, vol. 59, no. 2, pp. 629–638, 2011

    2011

  26. [26]

    Weighted SPICE: A unifying approach for hyperparameter-free sparse estimation,

    P. Stoica, D. Zachariah, and J. Li, “Weighted SPICE: A unifying approach for hyperparameter-free sparse estimation,”Digital Signal Processing, vol. 33, pp. 1–12, 2014

    2014

  27. [27]

    Gradient descent provably optimizes over-parameterized neural networks,

    S. S. Du, X. Zhai, B. P ´oczos, and A. Singh, “Gradient descent provably optimizes over-parameterized neural networks,” inInternational Conference on Learning Representations (ICLR), 2019. [Online]. Available: https://openreview.net/forum?id=S1eK3i09YQ

    2019

  28. [28]

    Loss landscapes and optimization in over-parameterized non-linear systems and neural networks,

    C. Liu, L. Zhu, and M. Belkin, “Loss landscapes and optimization in over-parameterized non-linear systems and neural networks,”Applied and Computational Harmonic Analysis, vol. 59, pp. 85–116, 2022

    2022

  29. [29]

    Overparameterized nonlinear learn- ing: Gradient descent takes the shortest path?

    S. Oymak and M. Soltanolkotabi, “Overparameterized nonlinear learn- ing: Gradient descent takes the shortest path?” inProceedings of the 36th International Conference on Machine Learning (ICML), ser. Proceedings of Machine Learning Research, vol. 97. PMLR, 2019, pp. 4951–4960

    2019

  30. [30]

    A local Polyak-Łojasiewicz and descent lemma of gradient descent for overparametrized linear models,

    Z. Xu, H. Min, S. Tarmoun, E. Mallada, and R. Vidal, “A local Polyak-Łojasiewicz and descent lemma of gradient descent for overparametrized linear models,” 2025. [Online]. Available: https://arxiv.org/abs/2505.11664

    Pith/arXiv arXiv 2025

  31. [31]

    When are nonconvex problems not scary?

    J. Sun, Q. Qu, and J. Wright, “When are nonconvex problems not scary?” arXiv preprint arXiv:1510.06096, 2015

    Pith/arXiv arXiv 2015

  32. [32]

    Boyd and L

    S. Boyd and L. Vandenberghe,Convex Optimization. Cambridge University Press, 2004

    2004

  33. [33]

    Block acceleration without momentum: On optimal stepsizes of block gradient descent for least-squares,

    L. Peng and W. Yin, “Block acceleration without momentum: On optimal stepsizes of block gradient descent for least-squares,” arXiv preprint arXiv:2405.16020, 2024. [Online]. Available: https: //arxiv.org/abs/2405.16020

    Pith/arXiv arXiv 2024

  34. [34]

    On the convergence of block coordinate descent type methods,

    A. Beck and L. Tetruashvili, “On the convergence of block coordinate descent type methods,”SIAM Journal on Optimization, vol. 23, no. 4, pp. 2037–2060, 2013

    2037

  35. [35]

    Projected gradient descent for Toeplitz covariance estimation,

    B. B ¨ock, D. Semmler, B. Fesl, M. Baur, and W. Utschick, “Projected gradient descent for Toeplitz covariance estimation,”IEEE Transactions on Signal Processing, vol. 73, pp. 876–890, 2025

    2025

  36. [36]

    S. M. Kay,Fundamentals of Statistical Signal Processing: Estimation Theory. Englewood Cliffs, NJ: Prentice Hall PTR, 1993, vol. 1

    1993

  37. [37]

    Comparing KL divergence and MSE for covariance estimation in target detection,

    D. Busbib, T. Diskin, and A. Wiesel, “Comparing KL divergence and MSE for covariance estimation in target detection,” inIEEE Statistical Signal Processing Workshop (SSP), 2025, pp. 101–105. APPENDIXA PROOF OFTHEOREM1 Due to the theory of maximum likelihood estimation, ifS= Cthen bC( baNLL) =S=Cis clearly a global minimum. The main part of the proof is tha...

    2025