REVIEW 3 major objections 5 minor 13 references
Contributions to Robust and Efficient Methods for Analysis of High Dimensional Data
T0 review · 3 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash
Pith's one-line read Three advances: FFT mutual-information screening, an O(1/k) accelerated-gradient rule for SCAD/MCP, and a qGaussian mixed-effects model.
desk verdict Second manuscript is a real contribution; third has a load-bearing closure failure; first is useful software with weak framing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
Three mechanisms carry the argument. The first is the FFT-based kernel density estimator: replacing direct kernel density estimation with Silverman's fast Fourier convolution turns mutual-information estimation into an O(n log n) operation, which is what makes screening 150,000 features practical. The second is the complexity-bound hyperparameter rule for the nonconvex accelerated gradient method: the recursion alpha_{k+1} = 2/(1 + sqrt(1 + 4/alpha_k^2)) with delta1 = omega_k = 2/(3L_Psi) minimizes the algorithm's upper bound and keeps the worst-case rate O(1/k). The third is the qGaussian density (5.20), the unique maximizer of Tsallis entropy under q-moment constraints; it supplies a bell-
What would settle it
Generate data from a genuine qGaussian linear mixed model: draw the random effects and noise separately from qGaussian distributions, form their sum, and compare the empirical marginal density of that sum with density (5.20) at the fitted q and Sigma; a material mismatch, which Property 3 predicts, would show the proposed likelihood is not the marginal of that generative model.
Extended reading notes
Core claim
On the paper's own terms, the discoveries are: (i) mutual information computed from FFT-accelerated kernel density estimates is the leading screening statistic for continuous outcomes with nonlinear dependence, while binning-based mutual information leads for binary outcomes whose probability transform is nonlinear; on the preprocessed ABIDE fMRI data, screening roughly 150,000 features completes in practical time and the selected variables improve downstream prediction. (ii) For SCAD- and MCP-penalized linear and logistic models, the accelerated gradient algorithm becomes practically usable when its hyperparameters are set to delta1 = omega_k = omega = 2/(3L_Psi), delta_{k+1} = omega/alpha_
Load-bearing premise
The load-bearing premise of the third contribution is that a qGaussian random effect plus qGaussian error can be represented by the qGaussian likelihood (5.20); the paper's own closure property—independent qGaussian vectors are not qGaussian under linear combination—means this premise holds only in special cases.
Editorial extensions
If this is right
- Mutual-information screening can be used as a first pass on neuroimaging-scale data, preserving nonlinear signals that Pearson correlation would miss before downstream penalized models are fit.
- The hyperparameter rule (4.13)-(4.15) gives practitioners a default recipe for nonconvex accelerated gradient, with a worst-case O(1/k) guarantee, so SCAD/MCP sparse estimation no longer requires per-problem tuning.
- The proximal conjugate-gradient framework extends a class of numerical methods for finding equilibria of Lipschitz flows to composite nonsmooth objectives, so any globally convergent smooth solver with a line search can be given a proximal version.
- The qGaussian likelihood broadens linear mixed-effects modeling to heavy-tailed data; in simulations it recovers signals where the coordinate-descent baseline fails to converge within practical time limits.
Reading between the lines
- Beyond the paper: the qGaussian mixed-effects model is most defensible when read as a quasi-likelihood or as a reparameterized t-distribution model, because the paper's own property that independent qGaussian vectors are not closed under linear combination means a qGaussian random effect plus qGaussian error is not generically qGaussian.
- Beyond the paper: the complexity-bound hyperparameter rule likely transfers to other nonconvex composite objectives, including penalized generalized linear mixed models, since the derivation depends on smoothness constants rather than on the specific forms of SCAD and MCP.
- Beyond the paper: the mutual-information screening ranking could be piped directly into the qGaussian sparse model to form one end-to-end pipeline; the thesis describes the two tools as sequential stages but does not benchmark them as a single combined procedure.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This thesis is organized as three manuscripts. Manuscript 1 presents fastHDMI, a Python package for mutual-information-based variable screening using FFTKDE, kNN, binning, and Pearson correlation; it claims that FFTKDE-based MI is best for continuous nonlinear outcomes, binning-based MI is best for certain nonlinear binary outcomes, and that the method is computationally practical on neuroimaging-scale ABIDE data. Manuscript 2 adapts the Ghadimi--Lan nonconvex accelerated gradient method to SCAD/MCP-penalized linear and logistic models, proposes a hyperparameter choice based on minimizing a complexity upper bound, proves an O(1/N) rate, and reports faster convergence and better signal recovery than ncvreg. Manuscript 3 re-derives the multivariate qGaussian density from Tsallis entropy maximization and proposes a qGaussian linear mixed-effects model with a proximal Hager--Zhang conjugate gradient algorithm. The thesis claims to deliver robust and efficient tools for high-dimensional screening, sparse estimation, and correlated-data modeling.
Significance. Each manuscript has standalone value. Manuscript 1 provides a working software tool and a rare neuroimaging-scale comparison of MI estimators; the simulations preserve the real ABIDE covariate structure, and the code is publicly available. Manuscript 2 is already published in Statistics and Computing, ships reproducible code, and contributes a practical hyperparameter recipe, an O(1/N) convergence rate, and a broad simulation study. Manuscript 3 contains a detailed algebraic derivation of the multivariate qGaussian density and an original framework for proximal conjugate gradient methods. However, the central qGaussian mixed-effects model is internally inconsistent with Property 3 of Section 5.3, and the hyperparameter optimality in Manuscript 2 is proven only in the convex case. These issues substantially temper the generality of the claimed contributions.
major comments (3)
- [Manuscript 3, Sections 5.3 and 5.5] The central model is not internally consistent. Section 5.3 Property 3 states that independent qGaussian random vectors are not closed under linear combination, and Property 2 gives closure only under left multiplication by a fixed matrix. In the mixed-effects formulation y = Xβ + Zb + ε, the random effect b and error ε enter additively, so y is a linear combination of independent qGaussian vectors. Consequently the marginal density of y is generically not the qGaussian density (5.20), and maximizing (5.20) is not maximum likelihood for the stated generative model. The chapter must either provide a special condition under which the sum remains qGaussian, or abandon the random-effects generative interpretation and define (5.20) as a working/marginal likelihood. As written, the estimation, prediction, and any statistical guarantees attached to the 'mixed-effects' interpretation are not jus
- [Manuscript 2, Theorem 1 and Section 4.3.2] The optimality of the proposed hyperparameters is established only for h ≡ 0 in Theorem 1, while the application is to SCAD/MCP composite objectives with nonconvex smooth h given by (4.3)--(4.4). Conditions (4.8)--(4.9) guarantee convergence for these h, but minimizing the complexity bound in the convex specialization does not by itself minimize the nonconvex bound. The manuscript should state this limitation explicitly and either prove an analogue for nonzero h or characterize the nonconvex use as a heuristic supported by the reported simulations. As written, the theoretical 'optimal damping sequence' claim is convex-only, while the abstract's 'considerably faster' claim is empirical.
- [Manuscript 3, Remark 8 and Section 5.5] Remark 8 concedes that for q > 1, the density (5.20) is exactly a reparameterized multivariate t density. Combined with the closure failure described above, the claimed robustness contribution of a 'qGaussian mixed-effects model' reduces, in substance, to a t-likelihood with a particular covariance parameterization. The chapter should compare explicitly with existing t-distribution LMMs and identify what is new beyond the reparameterization and the proximal Hager--Zhang algorithm; otherwise the novelty claim is overstated.
minor comments (5)
- [Manuscript 1, Figures 3.1 and 3.2] The legend labels are inconsistent: Figure 3.1 uses 'MI based on kNN' while Figure 3.2 uses 'MI based on sklearn'. Please use consistent method names.
- [Manuscript 3, Algorithm 3] The input specification 'ρ ∈ (0, ρ^{-1})' appears to be a typo; from Lemmas 11--12 it should be 'ρ ∈ (0, L_{∇g}^{-1})'.
- [Manuscript 1, Section 3.2] The consistency claim for the MI estimators is made by invoking the continuous mapping theorem, but the required functional continuity and uniform convergence conditions are not stated. This is too informal for a theoretical claim; either provide a precise statement with conditions or cite a source.
- [Manuscript 3, Section 5.5] The notation 'qtrain' is used in the optimization subsection but is not explicitly defined in relation to the parameter q in (5.20). Clarify whether qtrain is the shape parameter q or a training-set-specific estimate.
- [General] There are several typographical slips, e.g., 'prepossessed' for 'preprocessed' in Manuscript 1, and 'Baghwat' inconsistently spelled in the author list. A careful proofread is needed.
Circularity Check
No significant circularity: the three manuscripts validate their proposals against simulated ground truth, held-out test data, and an external complexity criterion; Manuscript 3's weakness is internal model coherence, not circularity.
full rationale
I traced each claimed derivation chain. Manuscript 1 (fastHDMI) estimates mutual information from the data and evaluates variable-selection AUROC against simulated true signals; the case studies use held-out test sets, so there is no fit-then-predict conflation. Manuscript 2 chooses its hyperparameters (δ1=ωk=ω=2/(3LΨ), α recursion (4.13)) by minimizing the complexity upper bound of the Ghadimi–Lan algorithm, an external analytical criterion; convergence and signal recovery are then measured on independently simulated data, so the simulations are not forced by construction. Manuscript 3 re-derives the qGaussian density (5.20) from a stated Tsallis-entropy maximization problem using external mathematical results (Vignat et al.; Tsukada) and then proposes an optimization framework; the density is not fitted to the algorithm's outputs, and the Hager–Zhang proximal method is benchmarked through its own convergence analysis. The strongest concern in the thesis—the tension between Property 3 (independence is not closed under linear combination) and the qGaussian linear mixed-effects generative story—is a correctness/coherence issue, not a circular reduction: the likelihood is not defined in terms of the data-generating mixture, nor is any fitted parameter renamed as a prediction. Under the hard rules, internal inconsistency without constructional equivalence does not raise the circularity score.
Assumptions & free parameters
free parameters (4)
- q (shape parameter, 'qtrain') =
maximum likelihood on the training set (Section 5.5.2)
- sigma^2 (dispersion) =
maximum likelihood (Section 5.5.2)
- penalty hyperparameters lambda, a (SCAD), gamma (MCP) =
validation-set selection; 50-value lambda grid (Section 4.5.1)
- KDE bandwidth and bin count (Manuscript 1) =
Improved Sheather-Jones bandwidth; Birgé-Rozenholc optimal bins (Sections 3.2, A.1)
assumptions (6)
- standard math The Beta-function integral formula (5.24), attributed to M. Tsukada (2005), used to normalize the multivariate qGaussian density.
- standard math Strict concavity of the Tsallis objective (5.10) on the feasible set implies a unique maximizer.
- domain assumption Unconstrained sparse problems can be reduced to a bounded domain without changing the solution set.
- domain assumption The nonconvex smooth Lipschitz constants L_SCAD = 1/(a-1) and L_MCP = 1/gamma are negligible relative to the spectral norm of the design matrix in high dimensions.
- domain assumption The Rényi (1959), Reshef et al. (2011), and Speed (2011) association measures are deterministic monotone functions of mutual information.
- domain assumption Observations y can be assigned the qGaussian density (5.20) with a covariance inherited from a linear mixed-effects decomposition.
Cite this review
Pith. "Pith review of Contributions to Robust and Efficient Methods for Analysis of High Dimensional Data." pith.science (2026). https://pith.science/paper/IWYBQFQ4
@misc{pith2026250908155,
author = {Pith},
title = {Pith review of: Contributions to Robust and Efficient Methods for Analysis of High Dimensional Data},
year = {2026},
howpublished = {\url{https://pith.science/paper/IWYBQFQ4}},
note = {Machine review of arXiv:2509.08155}
}
read the original abstract
A ubiquitous feature of data of our era is their extra-large sizes and dimensions. Analyzing such high-dimensional data poses significant challenges, since the feature dimension is often much larger than the sample size. This thesis introduces robust and computationally efficient methods to address several common challenges associated with high-dimensional data. In my first manuscript, I propose a coherent approach to variable screening that accommodates nonlinear associations. I develop a novel variable screening method that transcends traditional linear assumptions by leveraging mutual information, with an intended application in neuroimaging data. This approach allows for accurate identification of important variables by capturing nonlinear as well as linear relationships between the outcome and covariates. Building on this foundation, I develop new optimization methods for sparse estimation using nonconvex penalties in my second manuscript. These methods address notable challenges in current statistical computing practices, facilitating computationally efficient and robust analyses of complex datasets. The proposed method can be applied to a general class of optimization problems. In my third manuscript, I contribute to robust modeling of high-dimensional correlated observations by developing a mixed-effects model based on Tsallis power-law entropy maximization and discussed the theoretical properties of such distribution. This model surpasses the constraints of conventional Gaussian models by accommodating a broader class of distributions with enhanced robustness to outliers. Additionally, I develop a proximal nonlinear conjugate gradient algorithm that accelerates convergence while maintaining numerical stability, along with rigorous statistical properties for the proposed framework.
Figures
Figures from the paper (13 more)
Reference graph
Works this paper leans on
-
[1999]
ISSN 0378-4266. 10.1016/s0378-4266(98)00134-4. Ernesto Pereda, Miguel García-Torres, Belén Melián-Batista, Soledad Mañas, Leopoldo Méndez, and Julián J. González. The blessing of dimensionality: Feature selection out- performs functional connectivity-based feature transformation to classify ADHD subjects from EEG patterns of phase synchronisation. PLOS ON...
arXiv 2018
-
[2001]
URL http://www.jstor.org/stable/3085904
ISSN 0162-1459. URL http://www.jstor.org/stable/3085904. Miaolin Fan and Chun-An Chou. Exploring stability-based voxel selection methods in MVPA using cognitive neuroimaging data: a comprehensive study. Brain Informatics , 3(3):193– 203, April 2016. 10.1007/s40708-016-0048-0. Elsa Santos Febles, Marlis Ontivero Ortega, Michell Valdés Sosa, and Hichem Sahl...
-
[2005]
10.1111/j.1467-9868.2005.00503.x
ISSN 13697412, 14679868. 10.1111/j.1467-9868.2005.00503.x. URL http://www .jstor.org/stable/3647580. Hui Zou and Runze Li. One-step sparse estimates in nonconcave penalized likelihood models. The Annals of Statistics , 36(4):1509–1533, 2008. ISSN 0090-5364. 10.48550/arxiv.0808 .1012. URL http://www.jstor.org/stable/25464679. 197
arXiv 2005
-
[2008]
10.1016/j.neuroimage.2008.06.037. Rahul Mazumder, Jerome H. Friedman, and Trevor Hastie. Sparsenet: Coordinate descent with nonconvex penalties. Journal of the American Statistical Association , 106(495):1125– 1138, 2011. ISSN 0162-1459. URL http://www.jstor.org/stable/23427579. Elizabeth Meckes. The eigenvalues of random matrices. IMAGE, the Bulletin of ...
-
[2009]
ISSN 0378-4371. 10.1016/j.physa.2008.11.001. C Vignat, A.O Hero III, and J.A Costa. About closedness by convolution of the tsal- lis maximizers. Physica A: Statistical Mechanics and its Applications , 340(1-3):147–152, September 2004. 10.1016/j.physa.2004.04.001. Peter M. Visscher, Naomi R. Wray, Qian Zhang, Pamela Sklar, Mark I. McCarthy, Matthew A. Brow...
-
[2010]
ISBN 3642083048. Brian C. Ross. Mutual information between discrete and continuous data sets. PLoS ONE, 9(2):e87357, February 2014. 10.1371/journal.pone.0087357. I. M. Ross. An optimal control theory for accelerated optimization. February 2019. 10.48550/ ARXIV.1902.09004. Arkaprava Roy. Nonparametric group variable selection with multivariate response for...
work page Pith review arXiv doi:10.48550/arxiv.1902.09004 2014
-
[2011]
ISSN 1095-9203. 10.1126/science.1215894. Björn Sprungk. Numerical methods for Bayesian inference in Hilbert spaces . Universitätsver- lag Chemnitz, Chemnitz, 2017. Elias M. Stein and Rami Shakarchi. Fourier Analysis: An Introduction , volume 1. Princeton University Press, Princeton, 15. druck edition, 2003. ISBN 9780691113845. Hier auch später erschienene...
-
[2013]
10.1214/13-AOS1159. Jeremy Watt. Machine learning refined . Cambridge University Press, New York, second edition edition, 2020. ISBN 1108690939. Includes bibliographical references and index. Eugene P. Wigner. Characteristic vectors of bordered matrices with infinite dimensions. The Annals of Mathematics , 62(3):548, November 1955. ISSN 0003-486X. 10.2307/1...
arXiv 2020
Show all 13 references
-
[2014]
Sangin Lee and Patrick Breheny
10.1137/130921428. Sangin Lee and Patrick Breheny. Strong rules for nonconvex penalties and their implications for efficient algorithms in high-dimensional regression. Journal of Computational and Graphical Statistics, 24(4):1074–1091, October 2015. 10.1080/10618600.2014.975231....
2015
-
[2018]
Uwe Helmke
10.1002/wics.1454. Uwe Helmke. Optimization and Dynamical Systems . Springer London, London, 1994. ISBN 9781447134671. 10.1007/978-1-4471-3467-1. M.R. Hestenes and E. Stiefel. Methods of conjugate gradients for solving linear systems. Journal of Research of the National Bureau...
1994 doi
-
[2020]
URL https://api.semanticscholar.org/CorpusID:202607166. John H. Hubbard and Beverly H. West. Differential Equations: A Dynamical Systems Approach. Springer New York, 1995. ISBN 9781461241928. 10.1007/978-1-4612-4192-8. Megan J. Olson Hunt, Lisa Weissfeld, Robert M. Boudreau, Ho...
1995
-
[2024]
10.1007/s11222-023-10371-8
ISSN 1573-1375. 10.1007/s11222-023-10371-8. Yi Yang and Hui Zou. A fast unified algorithm for solving group-lasso penalize learning problems. Statistics and Computing , 25(6):1129–1141, August 2014. 10.1007/s11222-014 -9498-5. 196 Donghyeon Yu, Joong-Ho Won, Taehoon Lee, Johan ...
2014
-
[3003]
Hui Zou and Trevor Hastie
10.1016/j.amc.2009.08.058. Hui Zou and Trevor Hastie. Regularization and variable selection via the elastic net. Journal of the Royal Statistical Society. Series B (Statistical Methodology) , 67(2):301–320, March
2009 doi
Reviewed August 4, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.