Pith. sign in

REVIEW 3 major objections 5 minor 85 references

Diagonally-Weighted Generalized Method of Moments Estimation for Gaussian Mixture Modeling

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that a diagonally weighted GMM — one weight per moment order, no matrix inversion — stays consistent and asymptotically normal, sits between MM and GMM in efficiency, and avoids forming moment tensors entirely.

desk verdict A useful algorithm undermined by an unproven efficiency claim and a false local-identifiability assumption in Theorem 4.1. read the letter →

arxiv 2507.20459 v1 pith:K2KTUZ3B submitted 2025-07-28 cs.LG cs.NAmath.NAmath.STstat.MEstat.MLstat.TH

Liu Zhang , Oscar Mickelin , Sheng Xu , Amit Singer This is my paper · ORCID
classification cs.LGcs.NAmath.NAmath.STstat.MEstat.MLstat.TH MSC 62F1262H3015A6965Y20
keywords generalizedmethodofmomentsGaussianmixturemodelsdiagonalweightingmoment-basedestimationlow-rankcovarianceheteroscedasticmixturesBellpolynomialsNyströmapproximation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes DGMM, a version of the generalized method of moments in which the weighting matrix is forced to be diagonal with a single shared weight per moment order, chosen as the best diagonal approximation of the optimal GMM weight matrix. The central claim is that this cheap, inversion-free weighting still yields an estimator that is consistent and asymptotically normal, with asymptotic variance sitting between that of the unweighted MM and the fully weighted GMM. For weakly separated heteroscedastic low-rank Gaussian mixtures, the paper further claims an algorithm that obtains the estimator without ever computing or storing the moment tensors, at a cost that grows polynomially in the rank instead of exponentially in the dimension. The paper also reports experiments in which DGMM attains smaller estimation error in mixing probabilities, centers, and covariances while running substantially faster than MM and GMM. A reader would care because moment-based estimation has been accurate but computationally prohibitive in high dimensions, and DGMM is an attempt to keep the accuracy while removing the bottleneck.

What carries the argument

The load-bearing object is the DGMM weighting matrix $W = \mathrm{diag}(w_1,\dots,w_1,\dots,w_L,\dots,w_L)$, a diagonal matrix in which one scalar $w_k$ is shared by all $d^k$ moment conditions of order $k$, defined as the minimizer of $\|WS - I\|_F^2$ over such block-constant diagonals; the first-order optimality condition gives $w_k = \frac{\sum_{i\in I_k} S_{ii}}{\sum_{i\in I_k}\sum_{j=1}^q S_{ij}^2}$, the trace-to-Frobenius ratio of the $k$-th moment block of the asymptotic covariance $S$. This form is what removes the $S^{-1}$ inversion step of classical GMM while retaining a weighted moment-matching objective. The computational machinery is the implicit moment representation: the Bell-polynomial identities in Proposition 2.8 convert model cumulants into the tensor norms and inner products $\alpha_k$, $\beta_{k,n}$, $\gamma_{k,n,n'}$, so the weights (Theorem 4.9) and objective gradients (Theorem 4.10) are computed without forming $M^{(k)}(\theta)$ or $y_n^{\otimes k}$; a Nyström approximation with $k$-means++ landmarks (Theorem 4.5, Lemma 4.6) handles the $\sum_{n'}\gamma_{k,n,n'}$ sums, whose rank is bounded by $\binom{R_{\max}+k-1}{k}$ (Lemma 4.7).

What would settle it

Evaluate the Jacobian $G(\theta^*)$ of the stacked moment conditions (2.21) at a ground-truth parameter whose two components are identical in distribution: the exchange direction — perturbing the two labels in opposite senses — leaves every expected moment unchanged, annihilating the Jacobian, so its rank is strictly below $p$ and the local identification assumption Theorem 4.1 requires fails for this model. A purely empirical check: simulate many datasets from a known well-separated two-component model, compute the DGMM estimate on each, and compare the sample covariance of the estimates with the formula in (4.1); a persistent mismatch would falsify the claimed asymptotic variance.

Watch

Extended reading notes

Core claim

The central discovery, stated as Theorem 4.1, is that a GMM estimator weighted by the Frobenius-optimal diagonal approximation of the full optimal weight matrix — block-constant weights $w_k = \frac{\sum_{i\in I_k} S_{ii}}{\sum_{i\in I_k}\sum_j S_{ij}^2}$ for each moment order $k$ — remains consistent and asymptotically normal under the standard regularity conditions, with asymptotic variance $$$V^{{(\mathrm{DGMM}}$)} = \left[\sum_{k=1}^L w_k $G_k^{{\mathsf T}}$G_k\right]^{-1}\left[\sum_{k,k'} w_k w_{k'} $G_k^{{\mathsf T}}$ S_{kk'} G_{k'}\right]\left[\sum_{k=1}^L w_k $G_k^{{\mathsf T}}$G_k\right]^{-1},$$ which the authors read as intermediate efficiency between MM and the optimally weighted GMM, with the gap shrinking when cross-order moment correlations are weak. The second discovery is algorithmic: for a weakly separated heteroscedastic low-rank Gaussian mixture, the weights and the gradient of the DGMM objective can be expressed through the scalar quantities $\alpha_k = \|M^{(k)}(\theta)\|^2$, $\beta_{k,n} = \langle M^{(k)}(\theta), y_n^{\otimes k}\rangle$, and $\gamma_{k,n,n'} = \langle y_n, y_{n'}\rangle^k$, computed via Bell-polynomial identities for cumulants plus a Nyström approximation of the kernel sums, so the estimator is obtained in $O(L^2K^2 + LK^2 dR_{\max}^2 + NLK + NK dR_{\max} + NLdm)$ operations rather than at the $O(d^L)$ cost of forming moment tensors.

Load-bearing premise

The whole theory leans on an assumption the paper states but does not verify: that the chosen moments identify the model uniquely and that the matrix of moment derivatives at the true parameter has full column rank; for a mixture model the label-permutation symmetry of the components makes that rank condition doubtful unless the parameter space is quotiented or constrained.

Editorial extensions

If this is right

  • If Theorem 4.1 holds, moment-based estimation of weakly separated heteroscedastic low-rank Gaussian mixtures inherits GMM's consistency and asymptotic normality while skipping the $O(d^{2L})$ work of estimating and inverting the full weight matrix $S$.
  • The complexity bound of Theorem 4.11 keeps the cost polynomial in $d$ and $R_{\max}$ for fixed $K$ and $L$, so moment orders $L \ge 3$ and dimensions in the hundreds become reachable where the paper's analysis shows MM and GMM are not.
  • In the reported experiments, DGMM beats both the unweighted MM and the fully weighted GMM in estimation error on mixing probabilities, centers, and covariances, and it avoids the numerical instability GMM incurs from inverting the large estimated weight matrix.
  • Section 6 argues the same guarantees hold for any parametric model satisfying the identification and regularity conditions, so DGMM is proposed as a general substitute for GMM whenever the number of moment conditions is large relative to the sample size.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: because the weights are scalar per order, DGMM pools information across all entries of an order; a refinement that keeps within-order block structure, or drops the cross-order blocks $S_{kk'}$ explicitly, would interpolate between DGMM and full GMM efficiency and could be tested on the same numerical setups.
  • Editorial inference: the Nyström landmark count $m$ is the practical lever of the algorithm; since Lemma 4.7 bounds the kernel rank by $\binom{R_{\max}+k-1}{k}$, small $R_{\max}$ and large $d$ make the method essentially linear in $N$, and one could measure empirically how $m$ must scale with $N$ and $R_{\max}$ to keep weight estimates stable.
  • Editorial inference: the identification gap is fixable in principle — quotienting the parameter space by label permutations or imposing an ordering constraint on the centers — and doing so would put Theorem 4.1 on solid ground; a natural companion study would run DGMM on the quotient space and compare realized variances with formula (4.1).
  • Editorial inference: because weights pool across all entries of an order, DGMM implicitly assumes comparable noise within each order; for strongly anisotropic mixtures an adaptive within-order grouping, say by the symmetry class of the tensor entries, is a concrete extension the paper leaves open.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces DGMM, a diagonally-weighted variant of GMM in which the weighting matrix is a block-constant diagonal matrix with one weight per moment order, chosen as the optimal diagonal approximation of the inverse asymptotic covariance matrix of the moment conditions. The estimator is applied to weakly separated heteroscedastic low-rank Gaussian mixtures (Model 1.1). The paper claims three types of contributions: statistical (Theorem 4.1: consistency, asymptotic normality, and an asymptotic variance formula said to give intermediate efficiency between MM and GMM), algorithmic (Theorems 4.3--4.11: computation of weights and gradients without forming moment tensors, using Bell polynomials and a Nyström approximation), and numerical (Section 5: DGMM achieves lower estimation error and shorter runtime than MM and GMM in experiments).

Significance. If the statistical claims were fully established, DGMM would be a practically valuable estimator: it avoids the large matrix inversion of GMM while retaining some efficiency gains over unweighted MM, and the proposed algebraic-complexity reductions are detailed and plausible. The computational complexity results and the numerical comparisons are useful and clearly presented. However, the central theoretical result is currently not proven for the actual model and estimator used in the paper: the identification assumptions under which Theorem 4.1 holds are neither verified nor satisfied under the paper's own parameterization, and the intermediate-efficiency claim is not derived. The algorithmic and empirical contributions are promising, but the statistical claims need substantial repair before the paper can be accepted.

major comments (3)
  1. [Section 2.4, Theorem 4.1] This is the load-bearing issue: the estimator run in Sections 3--5 is exactly the one whose asymptotic distribution is claimed in Theorem 4.1, but the hypotheses of that theorem are not satisfied by the model and parameterization used.
  2. [Abstract, Section 1.2, Theorem 4.1, Remark 4.2]
  3. [Section 4, Theorem 4.5, Theorem 4.11, Section 5]
minor comments (5)
  1. [Eq. (4.2), Appendix B Eq. (B.3)] Equation (4.2) appears to lack the usual 1/N normalization in the definition of S_{kk'}; as typeset, the limit diverges unless 'NX' is intended to denote (1/N)Σ_{n=1}^N. Please correct the definition so that it matches the standard GMM asymptotic variance matrix and the formula (2.25).
  2. [Table 1] The 'Asymptotic efficiency' row for DGMM should be updated consistently with the outcome of the intermediate-efficiency discussion; as written it repeats the unproven claim that DGMM achieves intermediate efficiency.
  3. [Model 1.1 and Section 5] The 'weakly separated' condition ∥Σ_j∥_F ≫ ∥µ_j∥_2 is stated as part of Model 1.1 and used to motivate the numerical regime, but it does not appear in any theorem or proof; please clarify whether it is needed for the statistical results or only for the experimental setup.
  4. [Section 5] The numerical section reports means and standard deviations over 7 runs but does not state how random seeds or initializations are controlled across methods; describing this would make the runtime and error comparisons reproducible.
  5. [Appendix B, proof of Theorem 4.5] The proof of Theorem 4.5 lists kernel k-means++ landmark selection from [66] and then a randomly pivoted Cholesky decomposition from [21]; please clarify the relationship between these two steps and why the error bound in Lemma 4.6 applies to the implemented procedure as described.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: DGMM theory is a standard conditional GMM argument and the only self-citation is non-load-bearing.

full rationale

The derivation chain is not circular. Theorem 4.1 is a direct application of the standard GMM consistency and asymptotic-normality theorem (Newey-McFadden [65, Thm. 2.6]) to the DGMM objective (3.4) with a block-diagonal weight matrix. The hypotheses used in the proof—compact parameter space, continuity of polynomial moment functions, and the global/local identification assumptions—are stated explicitly in Section 2.4 and re-checked in Appendix B, not derived from the conclusion. The weights are estimated from previous-step parameter estimates in (3.4), but this is textbook two-step GMM; the theorem's variance formula uses the population optimal diagonal weight W defined by (3.1)-(3.3), and no fitted parameter is renamed as a prediction. The only self-citation is [32] (co-author Sheng Xu) in Section 6, where it is cited merely for conceptual alignment with low-SNR likelihood/moment results ('this weighting strategy is conceptually aligned with recent results on likelihood and moment expansion in related low-SNR problems, e.g., [49, 33, 32]'); it is not used in any proof and is not load-bearing. The known rotational and permutation non-identifiability of the V_j parameterization in (2.19) means the blanket local-identification assumption may fail for the specific Model 1.1, but that concerns whether the hypotheses of Theorem 4.1 are satisfied, not whether the theorem reduces to its inputs. The paper itself states in Section 6 that the statistical properties hold only under those identification assumptions and standard regularity conditions. Thus no circular step is present.

Assumptions & free parameters 3 free parameters · 6 assumptions · 0 invented entities

The estimator rests on standard GMM regularity conditions, specific identifiability assumptions that are assumed rather than proven for Model 1.1, and the choice of tuning parameters L, m, and tau. No independent physical entities are introduced.

free parameters (3)
  • L (highest moment order) = L=3 for d=10 experiments; L=4 for d=100 experiments
    Chosen by the practitioner; the paper assumes a suitable L exists for identifiability and uses L=3 or 4 in simulations.
  • Nystrom landmark count m = not reported in experiments
    Accuracy of the DGMM weight computation depends on m; the paper only gives asymptotic error bounds, not the values used.
  • softmax temperature tau = not specified in text
    Controls peakedness of mixing probabilities and numerical stability; no value or tuning procedure is given for the numerical studies.
assumptions (6)
  • domain assumption Global identifiability: E[g(θ,y_n)]=0 iff θ=θ*
    Assumed in Section 2.4; not proven for Model 1.1, and label-switching symmetry may violate it without a quotient constraint.
  • domain assumption Local identifiability: rank(G(θ*))=p
    Assumed in Section 2.4; needed for non-singular asymptotic variance and for Theorem 4.1.
  • domain assumption Compact parameter space Θ containing true parameters
    Standard regularity condition invoked in the proof of Theorem 4.1, following [40, 65].
  • domain assumption K and Rmax are known
    The estimation problem assumes the number of components and maximum rank are given, as stated in Section 2.3.
  • domain assumption Polynomial spectral decay of Gram matrix H^(k) and clustering potential condition
    Needed for the Nystrom approximation error bound in Lemma 4.6; the paper states these are typical and observed for Model 1.1 but does not verify them for all settings.
  • standard math Standard regularity conditions such as finite moments and continuous differentiability
    Invoked in the proof of Theorem 4.1; standard for Gaussian mixture models.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Diagonally-Weighted Generalized Method of Moments Estimation for Gaussian Mixture Modeling." pith.science (2026). https://pith.science/paper/K2KTUZ3B

@misc{pith2026250720459,
  author       = {Pith},
  title        = {Pith review of: Diagonally-Weighted Generalized Method of Moments Estimation for Gaussian Mixture Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/K2KTUZ3B}},
  note         = {Machine review of arXiv:2507.20459}
}
read the original abstract

Since Pearson [Philosophical Transactions of the Royal Society of London. A, 185 (1894), pp. 71-110] first applied the method of moments (MM) for modeling data as a mixture of one-dimensional Gaussians, moment-based estimation methods have proliferated. Among these methods, the generalized method of moments (GMM) improves the statistical efficiency of MM by weighting the moments appropriately. However, the computational complexity and storage complexity of MM and GMM grow exponentially with the dimension, making these methods impractical for high-dimensional data or when higher-order moments are required. Such computational bottlenecks are more severe in GMM since it additionally requires estimating a large weighting matrix. To overcome these bottlenecks, we propose the diagonally-weighted GMM (DGMM), which achieves a balance among statistical efficiency, computational complexity, and numerical stability. We apply DGMM to study the parameter estimation problem for weakly separated heteroscedastic low-rank Gaussian mixtures and design a computationally efficient and numerically stable algorithm that obtains the DGMM estimator without explicitly computing or storing the moment tensors. We implement the proposed algorithm and empirically validate the advantages of DGMM: in numerical studies, DGMM attains smaller estimation errors while requiring substantially shorter runtime than MM and GMM. The code and data will be available upon publication at https://github.com/liu-lzhang/dgmm.

Figures

Figures reproduced from arXiv: 2507.20459 by the authors.

Figure 1
Figure 1. 1000 i.i.d. samples from an example of Model 1.1: Y ∈ R 2 ∼ 0.5N (µ1, Σ1) + 0.4N (µ2, Σ2) + 0.1N (µ3, Σ3), where rank Σ1 = rank Σ2 = 1,rank Σ3 = 2. The distribution of Y is a mixture of two one-dimensional Gaussians and a two-dimensional Gaussian. 1.2. Contributions. We introduce a variant of GMM — the diagonally-weighted GMM (DGMM) — which uses the optimal diagonal approximation of the full GMM weighting matrix to … view at source ↗
Figure 2
Figure 2. (2a) shows a scatter plot of the data randomly sampled from a small-scale example [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗
Figure 3
Figure 3. (3a) shows a scatter plot of the data randomly sampled from a large-scale example [PITH_FULL_IMAGE:figures/full_fig_p016_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: (4a) shows a scatter plot of the data sampled from a small-scale example of [PITH_FULL_IMAGE:figures/full_fig_p017_4.png]
Figure 5
Figure 5. Figure 5: (5a) shows a scatter plot of the data randomly sampled from a large-scale example of [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

85 extracted references · 73 canonical work pages

  1. [1]

    A. Abas, T. Bendory, and N. Sharon , The generalized method of moments for multi-reference alignment, IEEE Transactions on Signal Processing, 70 (2022), pp. 1377–1388

    2022

  2. [2]

    Agostini, C

    D. Agostini, C. Améndola, and K. Ranestad , Moment identifiability of homoscedastic Gaussian mixtures, Foundations of Computational Mathematics, 21 (2021), pp. 695–724

    2021

  3. [3]

    J. G. Altonji and L. M. Segal , Small-sample bias in GMM estimation of covariance structures, Journal of Business & Economic Statistics, 14 (1996), pp. 353–366

    1996

  4. [4]

    Anandkumar, R

    A. Anandkumar, R. Ge, D. J. Hsu, S. M. Kakade, M. Telgarsky, et al. , Tensor decompositions for learning latent variable models., Journal of Machine Learning Research, 15 (2014), pp. 2773–2832

    2014

  5. [5]

    Anderson, M

    J. Anderson, M. Belkin, N. Goyal, L. Rademacher, and J. Voss , The more, the merrier: The blessing of dimensionality for learning large Gaussian mixtures, in Conference on Learning Theory, PMLR, 2014, pp. 1135–1164

    2014

  6. [6]

    Arora and R

    S. Arora and R. Kannan , Learning mixtures of arbitrary Gaussians, in Proceedings of the Thirty-Third Annual ACM Symposium on Theory of Computing, 2001, pp. 247–257

    work page 2001

  7. [7]

    J. Ba, M. A. Erdogdu, T. Suzuki, Z. W ang, and D. Wu ,Learning in the presence of low-dimensional structure: A spiked random matrix perspective, Advances in Neural Information Processing Systems, 36 (2023), pp. 17420–17449

    work page 2023

  8. [8]

    Bach, Sharp analysis of low-rank kernel matrix approximations, in Conference on Learning Theory, PMLR, 2013, pp

    F. Bach, Sharp analysis of low-rank kernel matrix approximations, in Conference on Learning Theory, PMLR, 2013, pp. 185–209

    work page 2013

Show all 85 references
  1. [9]

    Bakshi, I

    A. Bakshi, I. Diakonikolas, H. Jia, D. M. Kane, P. K. Kothari, and S. S. Vempala , Robustly learning mixtures of k arbitrary Gaussians, in Proceedings of the 54th Annual ACM SIGACT Symposium on Theory of Computing, 2022, pp. 1234–1247

    2022

  2. [10]

    Belkin and K

    M. Belkin and K. Sinha , Polynomial learning of distribution families, in 2010 IEEE 51st Annual Symposium on Foundations of Computer Science, IEEE, 2010, pp. 103–112

    2010

  3. [11]

    E. T. Bell , Exponential polynomials, Annals of Mathematics, 35 (1934), pp. 258–277

    1934

  4. [12]

    Bhaskara, M

    A. Bhaskara, M. Charikar, A. Moitra, and A. Vijayaragha v an , Smoothed analysis of tensor decompositions, in Proceedings of the Forty-Sixth Annual ACM Symposium on Theory of Computing, 2014, pp. 594–603

    2014

  5. [13]

    C. M. Bishop , Mixture density networks, 1994. This manuscript is for review purposes only. 28 L. ZHANG, O. MICKELIN, S. XU, AND A. SINGER

    1994

  6. [14]

    C. M. Bishop , Neural Networks for Pattern Recognition, Oxford University Press, 1995

    1995

  7. [15]

    C. M. Bishop , Pattern Recognition and Machine Learning, Springer, 2006

    2006

  8. [16]

    Boutsidis, P

    C. Boutsidis, P. Drineas, and M. W. Mahoney , Unsupervised feature selection for thek-means clustering problem, Advances in Neural Information Processing Systems, 22 (2009)

    2009

  9. [17]

    S. C. Brubaker and S. Vempala , Isotropic PCA and affine-invariant clustering, in 2008 49th Annual IEEE Symposium on Foundations of Computer Science, IEEE, 2008, pp. 551–560

    2008

  10. [18]

    T. T. Cai, X. J. Jeng, and J. Jin , Optimal detection of heterogeneous and heteroscedastic mixtures, Journal of the Royal Statistical Society Series B: Statistical Methodology, 73 (2011), pp. 629–662

    2011

  11. [19]

    J. Cao, M. G. Genton, D. E. Keyes, and G. M. Turkiyyah , Exploiting low-rank covariance structures for computing high-dimensional normal and student-t probabilities , Statistics and Computing, 31 (2021), pp. 1–16

    2021

  12. [20]

    C. A. Charalambides , Enumerative Combinatorics, Chapman and Hall/CRC, 2018

    2018

  13. [21]

    Y. Chen, E. N. Epperly, J. A. Tropp, and R. J. Webber , Randomly pivoted Cholesky: Practical approximation of a kernel matrix with few entry evaluations, Communications on Pure and Applied Mathematics, 78 (2025), pp. 995–1041

    2025

  14. [22]

    R. N. Cochran and F. H. Horne , Statistically weighted principal component analysis of rapid scanning wavelength kinetics experiments, Analytical Chemistry, 49 (1977), pp. 846–853

    1977

  15. [23]

    Comon, G

    P. Comon, G. Golub, L.-H. Lim, and B. Mourrain , Symmetric tensors and symmetric tensor rank, SIAM Journal on Matrix Analysis and Applications, 30 (2008), pp. 1254–1279

    2008

  16. [24]

    Compiani and Y

    G. Compiani and Y. Kitamura , Using mixtures in econometric models: A brief review and some new results, The Econometrics Journal, 19 (2016), pp. C95–C127

    2016

  17. [25]

    Comtet , Advanced Combinatorics: The Art of Finite and Infinite Expansions, Springer Science & Business Media, 1974

    L. Comtet , Advanced Combinatorics: The Art of Finite and Infinite Expansions, Springer Science & Business Media, 1974

    1974

  18. [26]

    Cramér, Mathematical Methods of Statistics, vol

    H. Cramér, Mathematical Methods of Statistics, vol. 26, Princeton University Press, 1946

    1946

  19. [27]

    Dasgupta, Learning mixtures of Gaussians, in 40th Annual Symposium on Foundations of Computer Science (Cat

    S. Dasgupta, Learning mixtures of Gaussians, in 40th Annual Symposium on Foundations of Computer Science (Cat. No. 99CB37039), IEEE, 1999, pp. 634–644

    1999

  20. [28]

    Dasgupta and L

    S. Dasgupta and L. J. Schulman , A two-round variant of EM for gaussian mixtures, in Proceedings of the 16th Conference on Uncertainty in Artificial Intelligence, UAI ’00, San Francisco, CA, USA, 2000, Morgan Kaufmann Publishers Inc., p. 152–159

    2000

  21. [29]

    A. P. Dempster, N. M. Laird, and D. B. Rubin , Maximum likelihood from incomplete data via the EM algorithm, Journal of the Royal Statistical Society: Series B (Methodological), 39 (1977), pp. 1–22

    1977

  22. [30]

    Ding and X

    C. Ding and X. He , K-means clustering via principal component analysis, in Proceedings of the Twenty-First International Conference on Machine Learning, 2004, p. 29

    2004

  23. [31]

    Drineas, M

    P. Drineas, M. W. Mahoney, and N. Cristianini , On the Nyström method for approximating a Gram matrix for improved kernel-based learning, Journal of Machine Learning Research, 6 (2005), pp. 2153–2175

    2005

  24. [32]

    Z. F an, R. R. Lederman, Y. Sun, T. W ang, and S. Xu , Maximum likelihood for high-noise group orbit estimation and single-particle cryo-EM, The Annals of Statistics, 52 (2024), pp. 52–77

    2024

  25. [33]

    Z. F an, Y. Sun, T. W ang, and Y. Wu , Likelihood landscape and maximum likelihood estimation for the discrete orbit recovery model, Communications on Pure and Applied Mathematics, 76 (2023), pp. 1208–1302

    2023

  26. [34]

    Fraley and A

    C. Fraley and A. E. Raftery , Model-based clustering, discriminant analysis, and density estimation, Journal of the American Statistical Association, 97 (2002), pp. 611–631

    2002

  27. [35]

    R. Ge, Q. Huang, and S. M. Kakade , Learning mixtures of Gaussians in high dimensions , in Proceedings of the Forty-Seventh Annual ACM Symposium on Theory of Computing, 2015, pp. 761–770

    2015

  28. [36]

    S. Gong, V. N. Boddeti, and A. K. Jain , On the intrinsic dimensionality of image representations, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019, pp. 3987–3996

    2019

  29. [37]

    B. Guo, J. Nie, and Z. Yang , Learning diagonal Gaussian mixture models and incomplete tensor decompositions, Vietnam Journal of Mathematics, (2022), pp. 1–26

    2022

  30. [38]

    Hackbusch, Numerical Tensor Calculus, Acta Numerica, 23 (2014), pp

    W. Hackbusch, Numerical Tensor Calculus, Acta Numerica, 23 (2014), pp. 651–742

    2014

  31. [39]

    A. R. Hall , Generalized method of moments, Handbook of Research Methods and Applications in This manuscript is for review purposes only. DIAGONALLY-WEIGHTED GENERALIZED METHOD OF MOMENTS 29 Empirical Macroeconomics, (2013), pp. 313–333

    2013

  32. [40]

    L. P. Hansen , Large sample properties of generalized method of moments estimators, Econometrica, 50 (1982), pp. 1029–1054

    1982

  33. [41]

    L. P. Hansen, J. Heaton, and A. Yaron , Finite-sample properties of some alternative GMM estimators, Journal of Business & Economic Statistics, 14 (1996), pp. 262–280

    1996

  34. [42]

    Healy and M

    M. Healy and M. Westmacott , Missing values in experiments analysed on automatic computers, Journal of the Royal Statistical Society: Series C (Applied Statistics), 5 (1956), pp. 203–206

    1956

  35. [43]

    Hinton, O

    G. Hinton, O. Vinyals, and J. Dean , Distilling the knowledge in a neural network, arXiv preprint arXiv:1503.02531, (2015)

    2015 arXiv

  36. [44]

    Hong and L

    D. Hong and L. Balzano , Optimal sample acquisition for optimally weighted PCA from heterogeneous quality sources, IEEE Signal Processing Letters, (2025)

    2025

  37. [45]

    D. Hong, F. Yang, J. A. Fessler, and L. Balzano , Optimally weighted PCA for high-dimensional heteroscedastic data, SIAM Journal on Mathematics of Data Science, 5 (2023), pp. 222–250

    2023

  38. [46]

    Hsu and S

    D. Hsu and S. M. Kakade , Learning mixtures of spherical Gaussians: Moment methods and spectral decompositions, in Proceedings of the 4th Conference on Innovations in Theoretical Computer Science, 2013, pp. 11–20

    2013

  39. [47]

    E. Jang, S. Gu, and B. Poole , Categorical reparameterization with Gumbel-softmax, arXiv preprint arXiv:1611.01144, (2016)

    2016 arXiv

  40. [48]

    A. T. Kalai, A. Moitra, and G. V aliant , Efficiently learning mixtures of two Gaussians, in Proceedings of the Forty-Second ACM Symposium on Theory of Computing, 2010, pp. 553–562

    2010

  41. [49]

    Katsevich and A

    A. Katsevich and A. S. Bandeira , Likelihood maximization and moment matching in low SNR Gaussian mixture models, CommunicationsonPureandAppliedMathematics, 76(2023), pp.788–842

    2023

  42. [50]

    C. G. Khatri, Some results for the singular normal multivariate regression models, Sankhy¯ a: The Indian Journal of Statistics, Series A, 30 (1968), pp. 267–280

    1968

  43. [51]

    Knill , Cauchy–Binet for pseudo-determinants, Linear Algebra and Its Applications, 459 (2014), pp

    O. Knill , Cauchy–Binet for pseudo-determinants, Linear Algebra and Its Applications, 459 (2014), pp. 522–547

    2014

  44. [52]

    T. G. Kolda and B. W. Bader , Tensor decompositions and applications, SIAM Review, 51 (2009), pp. 455–500

    2009

  45. [53]

    Kumar, J

    S. Kumar, J. Ying, J. V. d. M. Cardoso, and D. P. Palomar , A unified framework for structured graph learning via spectral constraints, Journal of Machine Learning Research, 21 (2020), pp. 1–60

    2020

  46. [54]

    Landsberg , Tensors: Geometry and Applications, vol

    J. Landsberg , Tensors: Geometry and Applications, vol. 128 of Graduate Studies in Mathematics, American Mathematical Society, 2011

    2011

  47. [55]

    Ledoit and M

    O. Ledoit and M. Wolf , A well-conditioned estimator for large-dimensional covariance matrices, Journal of Multivariate Analysis, 88 (2004), pp. 365–411

    2004

  48. [56]

    Lindberg, C

    J. Lindberg, C. Améndola, and J. I. Rodriguez , Estimating Gaussian mixtures using sparse polynomial moment systems, SIAM Journal on Mathematics of Data Science, 7 (2025), pp. 224–252

    2025

  49. [57]

    Liu and A

    A. Liu and A. Moitra , Robustly learning general mixtures of Gaussians, Journal of the ACM, 70 (2023), pp. 1–53

    2023

  50. [58]

    D. C. Liu and J. Nocedal , On the limited memory BFGS method for large scale optimization, Mathematical Programming, 45 (1989), pp. 503–528

    1989

  51. [59]

    Lyu and D

    Z. Lyu and D. Xia , Optimal estimation and computational limit of low-rank Gaussian mixtures, The Annals of Statistics, 51 (2023), pp. 646–667

    2023

  52. [60]

    C. J. Maddison, A. Mnih, and Y. W. Teh , The concrete distribution: A continuous relaxation of discrete random variables, arXiv preprint arXiv:1611.00712, (2016)

    2016 arXiv

  53. [61]

    A. G. McKendrick , Applications of mathematics to medical problems, Proceedings of the Edinburgh Mathematical Society, 44 (1925), pp. 98–130

    1925

  54. [62]

    McLachlan and D

    G. McLachlan and D. Peel , Finite Mixture Models, Wiley Series in Probability and Statistics, Wiley, 2004

    2004

  55. [63]

    Moitra and G

    A. Moitra and G. V aliant, Settling the polynomial learnability of mixtures of Gaussians, in 2010 IEEE 51st Annual Symposium on Foundations of Computer Science, IEEE, 2010, pp. 93–102

    2010

  56. [64]

    Newcomb , A generalized theory of the combination of observations so as to obtain the best result, American Journal of Mathematics, 8 (1886), pp

    S. Newcomb , A generalized theory of the combination of observations so as to obtain the best result, American Journal of Mathematics, 8 (1886), pp. 343–366

  57. [65]

    W. K. Newey and D. McF adden , Large sample estimation and hypothesis testing, in Handbook of Econometrics, vol. 4, Elsevier, 1994, pp. 2111–2245. This manuscript is for review purposes only. 30 L. ZHANG, O. MICKELIN, S. XU, AND A. SINGER

    1994

  58. [66]

    Oglic and T

    D. Oglic and T. Gärtner , Nyström method with kernel k-means++ samples as landmarks, in International Conference on Machine Learning, PMLR, 2017, pp. 2652–2660

    2017

  59. [67]

    Pearson , Contributions to the mathematical theory of evolution, Philosophical Transactions of the Royal Society of London

    K. Pearson , Contributions to the mathematical theory of evolution, Philosophical Transactions of the Royal Society of London. A, 185 (1894), pp. 71–110

  60. [68]

    Peccati and M

    G. Peccati and M. S. Taqqu , Wiener Chaos: Moments, Cumulants and Diagrams, vol. 1 of Bocconi & Springer Series, Springer Milan, 2011

    2011

  61. [69]

    J. M. Pereira, J. Kileel, and T. Kolda , Tensor moments of Gaussian mixture models: Theory and applications, arXiv preprint arXiv:2202.06930, (2022)

    2022 arXiv

  62. [70]

    Permuter, J

    H. Permuter, J. Francos, and I. Jermyn , Gaussian mixture models of texture and colour for image database retrieval, in2003IEEEInternationalConferenceonAcoustics, Speech, andSignalProcessing,

  63. [71]

    P. Pope, C. Zhu, A. Abdelkader, M. Goldblum, and T. Goldstein , The intrinsic dimension of images and its impact on learning, arXiv preprint arXiv:2104.08894, (2021)

    2021 arXiv

  64. [72]

    C. R. Rao , Multivariate analysis, in Multivariate Analysis, John Wiley & Sons, Ltd, 1973, ch. 8, pp. 516–604

    1973

  65. [73]

    D. A. Reynolds and R. C. Rose ,Robust text-independent speaker identification using Gaussian mixture speaker models, IEEE Transactions on Speech and Audio Processing, 3 (1995), pp. 72–83

    1995

  66. [74]

    Roodman, A note on the theme of too many instruments, Oxford Bulletin of Economics and Statistics, 71 (2009), pp

    D. Roodman, A note on the theme of too many instruments, Oxford Bulletin of Economics and Statistics, 71 (2009), pp. 135–158

    2009

  67. [75]

    Sriv asta v a and C

    M. Sriv asta v a and C. Khatri, An Introduction to Multivariate Statistics, North-Holland/New York, 1979

    1979

  68. [76]

    J. P. Stanczuk, G. Batzolis, T. Deveney, and C.-B. Schönlieb , Diffusion models encode the intrinsic dimension of data manifolds, in Proceedings of the Forty-First International Conference on Machine Learning, 2024

    2024

  69. [77]

    Tamuz, T

    O. Tamuz, T. Mazeh, and S. Zucker , Correcting systematic effects in a large set of photometric light curves, Monthly Notices of the Royal Astronomical Society, 356 (2005), pp. 1466–1470

    2005

  70. [78]

    Ta veira Blomenhofer, Gaussian mixture identifiability from degree 6 moments, Algebraic Statistics, 16 (2024), pp

    A. Ta veira Blomenhofer, Gaussian mixture identifiability from degree 6 moments, Algebraic Statistics, 16 (2024), pp. 1–28

    2024

  71. [79]

    Teicher , Identifiability of finite mixtures , The Annals of Mathematical Statistics, (1963), pp

    H. Teicher , Identifiability of finite mixtures , The Annals of Mathematical Statistics, (1963), pp. 1265–1269

    1963

  72. [80]

    Titterington, A

    D. Titterington, A. Smith, and U. Makov , Statistical Analysis of Finite Mixture Distributions, Applied Section, Wiley, 1985

    1985

  73. [81]

    Vempala and G

    S. Vempala and G. W ang , A spectral algorithm for learning mixture models, Journal of Computer and System Sciences, 68 (2004), pp. 841–860

    2004

  74. [82]

    Williams and M

    C. Williams and M. Seeger , Using the Nyström method to speed up kernel machines, Advances in Neural Information Processing Systems, 13 (2000)

    2000

  75. [83]

    Zhang, J

    H. Zhang, J. Zhou, Y. Lu, M. Guo, P. W ang, L. Shen, and Q. Qu , The emergence of reproducibility and generalizability in diffusion models, arXiv preprint arXiv:2310.05264, (2024)

    2024 arXiv

  76. [84]

    R. Zhou, J. Ying, and D. P. Palomar , Covariance matrix estimation under low-rank factor model with nonnegative correlations, IEEE Transactions on Signal Processing, 70 (2022), pp. 4020–4030. This manuscript is for review purposes only

    2022

  77. [2003]

    (ICASSP ’03)., vol

    Proceedings. (ICASSP ’03)., vol. 3, IEEE, 2003, pp. III–569

    2003

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.