Pith. sign in

REVIEW 4 major objections 5 minor 67 references

Information-theoretic limits and approximate message-passing for high-dimensional time series

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read AR(1) time-series regression gets exact information limits

desk verdict Plausible and novel MI formula for block-AR(1) regression, but the lower-bound proof has a broken Jacobian step and the abstract overstates the MMSE result. read the letter →

arxiv 2501.13625 v2 pith:KPZILPTJ submitted 2025-01-23 cs.IT cond-mat.dis-nnmath.ITmath.STstat.TH

Daria Tieplova , Samriddha Lahiry , Jean Barbier This is my paper · ORCID
classification cs.ITcond-mat.dis-nnmath.ITmath.STstat.TH
keywords high-dimensionaltimeseriesstochasticregressionmutualinformationreplicamethodadaptiveinterpolationvectorapproximatemessagepassingKac-Murdock-Szegömatrixminimummean-squareerror
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper asks what can be learned about a dense high-dimensional signal from observations of a time series whose covariates follow an AR(1) process, when the number of features and the number of samples grow at the same rate. It proves that the normalized mutual information between the observations and the signal converges to the value of a finite-dimensional variational problem: the infimum over one set of parameters and the supremum over another of a potential built from the AR(1) correlation spectrum. It also proves a closed-form formula for the measurement MMSE and states a conjecture for the per-block MMSE. The rigorously proven part covers the mutual information and the measurement MMSE; with the additional block-MMSE conjecture, the description becomes complete.

What carries the argument

The load-bearing object is the replica-symmetric potential $i_{\mathrm{RS}}(r_1,r_2)$, whose two vector arguments act as control parameters for the block structure: $r_1$ couples to scalar denoising channels $\beta\mapsto\sqrt{r_{1,i}}\beta + Z$, while $r_2$ enters through the spectral density $\delta_i(\theta)$ of the Kac-Murdock-Szegö covariance matrix of each AR(1) column. The argument runs through adaptive interpolation: one interpolates between the original time-series channel and $k$ decoupled scalar channels, controls the derivatives via overlap concentration, and uses the known limiting eigenvalue distribution of KMS matrices to evaluate the log-det terms. The fixed-point equations (II.9) define the set $\Gamma$ of critical points, and the inf-sup formula selects the global minimum.

What would settle it

Run exact Bayesian inference on a small instance with $k=2$ eigenvalue blocks placed on opposite sides of the phase transition seen in Figure 2, compute the per-block MMSE by Monte Carlo, and compare it with $\tilde{r}_{2,i}$ at the global minimum of $i_{\mathrm{RS}}$. If the block MMSE deviates from $\tilde{r}_{2,i}$ while the measurement MMSE still follows (II.21), the conjecture (II.23) is false even though the mutual-information formula remains correct.

Watch

Extended reading notes

Core claim

For the stochastic regression model $Y_\mu = p^{-1/2} x_\mu^\top \beta_0 + Z_\mu$ with $x_{\mu+1} = A_p x_\mu + \xi_\mu$, where $A_p$ is diagonal with $k$ fixed eigenvalues, the paper establishes $\lim_{p\to\infty} i_p = \inf_{r_1\in[0,\infty)^k}\sup_{r_2\in[0,\rho]^k} i_{\mathrm{RS}}(r_1,r_2)$, with the replica-symmetric potential given by (II.6) and $\delta_i(\theta) = (1 - 2\lambda_i\cos\theta + \lambda_i^2)^{-1}$ the spectral density of the AR(1) column covariance. The proof, via adaptive interpolation, shows that the mutual information per parameter is governed by this low-dimensional potential even though the design matrix is not right-rotationally invariant. A second theorem proves the limiting measurement MMSE equals $\frac{\sigma^2}{\pi}\int_0^\pi \frac{\sum_i l_i\delta_i(\theta)\tilde{r}_{2,i}}{\sum_i l_i\delta_i(\theta)\tilde{r}_{2,i}+\sigma^2}\,d\theta$ and relates it to the block MMSEs. The per-block MMSE formula (II.23), asserting that each block's error equals the saddle-point value $\tilde{r}_{2,i}$, remains a conjecture, and the numerical experiments show VAMP matching these predictions away from phase transitions but becoming unstable near them.

Load-bearing premise

The unproven step is that the posterior error on each block settles at the single saddle-point value $\tilde{r}_{2,i}$; this needs the replica-symmetric global minimum to be unique, and it can fail where the system has competing optimal states, which is exactly where the experiments show the algorithm becoming unstable.

Editorial extensions

If this is right

  • For any fixed number $k$ of AR(1) eigenvalues, the exact asymptotic mutual information is computed by a $k$-dimensional variational problem, reducing the inference problem to a finite optimization.
  • The measurement MMSE is available through formula (II.21), requiring only the saddle point of $i_{\mathrm{RS}}$ rather than a full posterior computation.
  • With an arbitrary limiting eigenvalue distribution for $A_p$, Theorem II.2 gives the same kind of formula with functions $r_1(\lambda), r_2(\lambda)$ in place of vectors, extending the result to continuously many AR(1) components.
  • If the per-block conjecture (II.23) holds, the posterior error on each block of coefficients is asymptotically $\tilde{r}_{2,i}$, giving a complete block-by-block description of estimation limits.
  • The empirical VAMP results indicate that a practical algorithm can reach the predicted MMSE outside phase-transition regions even without right rotational invariance.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: because only the spectral density $\delta_i(\theta)$ enters the formula, the same variational structure should also hold for other stationary Gaussian processes whose column covariance is asymptotically Toeplitz, such as ARMA processes; this is not tested in the paper.
  • Editorial extension: the observed VAMP instability exactly at phase transitions suggests the inf-sup formula can have multiple competing global minima; checking whether the block MMSE is discontinuous there would settle the conjecture and could inform when spectral initialization is needed.
  • Editorial extension: the same adaptive-interpolation proof likely extends to generalized linear observations on top of the AR(1) design, because the interpolation step decouples the temporal correlation from the likelihood.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper studies Bayesian inference in a high-dimensional stochastic regression model whose design matrix is generated by a diagonal AR(1) dynamics, with the number of features growing proportionally to the number of observations. The main result (Theorem II.1) is a variational formula for the normalized mutual information between the observations and a dense signal, proved by the adaptive interpolation method for the case where the diagonal transition matrix has k distinct eigenvalues; Theorem II.2 extends this to general eigenvalue distributions by a Wasserstein-approximation argument. Theorem II.3 gives a formula for the measurement MMSE and relates it to the per-block MMSEs. The paper also presents numerical experiments suggesting that VAMP achieves near-Bayes-optimal performance in this non-rotationally-invariant setting, and it transparently labels the per-block MMSE formula (II.23) as a conjecture.

Significance. If the main theorem is rigorously established, the paper provides a first single-letter formula for the information content of a high-dimensional AR(1) time series about a dense signal, going beyond the i.i.d. and right-rotationally-invariant designs previously treated in the literature. The reduction of the known λ=0 and λ=λI cases to prior results is a useful sanity check, and the extension to general diagonal matrices by approximating the eigenvalue distribution is an elegant step. The numerical VAMP study is also valuable despite the lack of theoretical guarantees. However, the proof of the central lower bound contains a broken Jacobian-derivative step, and several load-bearing concentration estimates are delegated to prior works without verification. The paper is therefore not yet a complete rigorous treatment, although the overall approach and the claimed formulas are plausible.

major comments (4)
  1. [Section IV-A, Lemma IV.5 (Eqs. (IV.25)-(IV.26))] The Liouville lower bound used to justify the change of variables from ε to R(t, ε) is not established. In the ODE (IV.25), F1,i is defined as A_i(ρ−E⟨Q_1⟩,...,ρ−E⟨Q_k⟩), so it depends on R1 only through the posterior overlaps E⟨Q_i⟩; the displayed derivative ∂R1,i F1 = c/(πk) ∫ δ_i²(θ)/(k^{-1}Σ_j δ_j(θ)R1,j + σ²) is not the partial derivative of F1 with respect to R1,i, and it does not follow from (IV.26). In fact the natural derivative of A_i with respect to its own argument would be negative and would contain a squared denominator. Since this positive Jacobian is the stated reason for applying Lemma IV.3, the liminf half of Theorem II.1 is not rigorously proved as written; the calculation must be corrected or replaced by a valid estimate of the Jacobian determinant.
  2. [Section IV-A, Lemma IV.2] The overlap concentration lemma is explicitly proved only in outline: the text says 'we outline the steps and omit the details' and asserts that inequalities (IV.18)-(IV.20) follow from the proofs in [4], [5]. This lemma is load-bearing for the fundamental identity Lemma IV.3 and hence for both bounds in Theorem II.1. The transfer from the models in [4], [5] to the present block-KMS design is not automatic, and the lemma also assumes the Jacobian regularity that is the subject of the previous comment. The full proof, or a precise statement of which results in [4], [5] apply verbatim and why, must be supplied.
  3. [Appendix E, estimates (E.9)-(E.11)] The proof of Lemma IV.7 delegates the crucial concentration estimates to prior work: (E.9) is said to follow from the 'same proof as Lemma 9.1 of [7]', (E.10) is declared 'equivalent to Lemma 9.2 of [7]', and only (E.14) receives a new argument because the independence used in [7] fails. Since the present design is neither i.i.d. nor right-rotationally invariant, the transfer of these lemmas requires verification of their hypotheses or a self-contained proof. Without (E.9)-(E.11), the proof of Theorem II.3 is incomplete.
  4. [Section II-B and Section III] The abstract and introduction claim derivation of 'minimum mean-square errors', but the rigorously proven statement is the measurement MMSE (II.21) together with the relation (II.22); the per-block and signal MMSEs in (II.23) are explicitly conjectural and rely on replica symmetry and uniqueness of the global minimizer of iRS on Γ. The numerical experiments in Section III compare VAMP against the conjectured curve (II.23), so the match is evidence for the conjecture rather than a proof of it. The text should be revised to state this limitation in the abstract and in the discussion of Figures 2 and 3, and the conditions under which the derivative in (II.20) can be interchanged with the variational formula should be stated.
minor comments (5)
  1. [Figure 2 caption] The caption appears to describe the panels inconsistently: it refers to 'On the right MMSE versus cN' and 'On the left we see MMSE ... versus 1/σ²', while the body text refers to Figure 2a and Figure 2b in the opposite order. Please align the caption with the actual panel layout.
  2. [Section IV-A, Step 4] The sentence 'This allows as to apply Lemma IV.3' contains a typo and should read 'This allows us to apply Lemma IV.3'.
  3. [Section IV-A, proof of Lemma IV.1] In the term E[⟨(r2,i(t) − (ρ − Qi))Z^T Λ_i,N u_t⟩], the placement of Qi inside the Gibbs bracket while r2,i(t) is deterministic should be clarified; it is currently ambiguous which quantities are quenched and which are averaged over the posterior.
  4. [Appendix D, Lemma D.5] The bounded-difference proof is only sketched: the text states that showing ψ′(s) ≤ Cp^{-1} would imply the result, but the bound on ψ′(s) is not displayed. Please complete the argument or give a precise reference.
  5. [Appendix B] The statement that KMS matrices 'asymptotically share the same eigenspace' is used in (A.37) and in Appendix E, but it is stated without proof or a precise reference. A formal statement of the asymptotic joint eigenvalue distribution would make the argument easier to verify.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the replica formula is derived, not assumed; self-citations provide only the established interpolation technique.

full rationale

The paper's central variational formula (Theorem II.1) is not an input: the potential iRS is defined in (II.6), and the adaptive-interpolation lemmas (IV.3-IV.5) are used to prove both the upper and lower bounds, while the special-case reductions to [3] and [8] are cross-checks. The citations to the authors' earlier adaptive-interpolation papers [4], [5] and to [8] supply a general proof method and concentration arguments, not the block-diagonal AR(1) result itself; those cited results are independently established and do not contain the target formula, so this is not load-bearing self-citation. The block-MMSE formula (II.23) is explicitly labeled a conjecture and is not passed off as a derived prediction, and the VAMP comparisons are empirical benchmarks rather than fitted parameters. The measurement-MMSE formulas in Theorem II.3 follow from the I-MMSE relation and a spectral trace computation, not from circular definitions. A genuine proof concern, though not a circularity, is that the positivity of the Jacobian in Lemma IV.5 is asserted via a derivative ∂_{R1,i} F1 that does not match the definition of F1 in (IV.25)-(IV.26); this affects the rigor of the liminf half of Theorem II.1 but does not make the derivation equivalent to its inputs.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The central MI formula has no fitted constants: all model hyperparameters, including the prior, noise variance, eigenvalue distribution, and sampling ratio, are inputs from the problem setting. The variational parameters r1 and r2 are optimization variables, not free parameters fitted to data. The proof relies on standard KMS spectral theory and on one unproven conjecture, the per-block MMSE, which is used in the empirical section.

assumptions (6)
  • standard math The eigenvalue distribution of Kac-Murdock-Szegő matrices converges to the density δ_i(θ) = (1 - 2λ_i cos θ + λ_i^2)^{-1}, and sums of such matrices are asymptotically simultaneously diagonalizable.
    Used to pass from finite-N spectral sums to θ-integrals in (II.6), (A.37), and throughout the proof of Theorem II.1. This is a known result from Grenander-Szegő [27] and Trench [60], stated in Appendix B.
  • domain assumption The matrix A_p is diagonal with eigenvalues in [0,1) (assumptions h1-h2), with fixed block proportions |I_i|/p -> l_i > 0 in Theorem II.1.
    This defines the stochastic regression model (I.1)-(I.3). It excludes non-diagonal and negative-eigenvalue AR dynamics; the entire KMS spectral analysis relies on this diagonal, nonnegative-eigenvalue structure.
  • domain assumption The prior P0 has compact support, i.i.d. entries, known variance ρ, and the Bayesian-optimal setting holds (prior and noise distribution known).
    The posterior (II.1), the MMSE computations, and the concentration lemmas in Appendix D all assume this. Compact support is used directly in the overlap concentration bounds.
  • domain assumption The AR(1) process is stationary with |λ_i|<1, so each column of the design matrix has covariance matrix Λ_i,N of (I.7).
    Stationarity gives the variance formula (I.5) and the Toeplitz structure (I.7). This is the defining model assumption and is used at the very start of the analysis.
  • ad hoc to paper The per-block MMSE conjecture (II.23): lim mmse(i) = r2,i at the unique global minimum of iRS on Γ.
    Not proven. Used for the paper's MMSE claim and for comparing VAMP to the 'theoretical' MMSE in Section III. Depends on replica symmetry and uniqueness of the saddle point, which can fail at first-order phase transitions.
  • standard math The limit exchange in Theorem II.2 (taking k to infinity after p to infinity) and uniform convergence of the potential on Lipschitz function classes hold.
    Sketch in Section IV-C relies on Wasserstein-2 convergence of the empirical spectral measures and on uniform convergence of the replica potential. The argument is stated as straightforward but is not fully detailed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Information-theoretic limits and approximate message-passing for high-dimensional time series." pith.science (2026). https://pith.science/paper/KPZILPTJ

@misc{pith2026250113625,
  author       = {Pith},
  title        = {Pith review of: Information-theoretic limits and approximate message-passing for high-dimensional time series},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/KPZILPTJ}},
  note         = {Machine review of arXiv:2501.13625}
}
read the original abstract

High-dimensional time series appear in many scientific setups, demanding a nuanced approach to model and analyze the underlying dependence structure. Theoretical advancements so far often rely on stringent assumptions regarding the sparsity of the underlying signal. In non-sparse regimes, analyses have primarily focused on linear regression models with the design matrix having independent rows. In this paper, we expand the scope by investigating a high-dimensional time series model wherein the number of features grows proportionally to the number of sampling points, without assuming sparsity in the signal. Specifically, we consider the stochastic regression model and derive a single-letter formula for the normalized mutual information between observations and the signal, as well as for minimum mean-square errors. We also empirically study the vector approximate message passing VAMP algorithm and show that, despite the lack of theoretical guarantees, its performance for inference in our time series model is robust and often statistically optimal.

Figures

Figures reproduced from arXiv: 2501.13625 by the authors.

Figure 1
Figure 1. On the left MMSE versus N/p for Gaussian prior. We take p = 2100, and Ap = 0 (red) and Ap = diag(0.9, 0.7, 0.5, 0.3, 0.1) (blue). On the right we see models with β drawn from Rademacher prior with p = 2100, σ 2 = 0.1. Blue line represents theoretically obtained MMSE for the case with i.i.d. Gaussian design matrix (Ap = 0) and on the red - the case of right rotation invariant design matrix (Ap = 0.9Ip). Bars represen… view at source ↗
Figure 2
Figure 2. On the right MMSE versus cN = N/p for models with Rademacher prior, σ 2 = 0.1, p = 2100, and two different choices of Ap. Continuous lines represent the theoretically obtained MMSE, (II.23), while the bars show the span of MSE obtained from the VAMP algorithm for 50 instances of the problem. On the left we see MMSE given by (II.23) versus 1/σ2 (blue), and MSE of VAMP averaged over 50 instances with signal with entri… view at source ↗
Figure 3
Figure 3. YMMSE versus cN = N/p for models with Rademacher prior, σ 2 = 0.1, p = 2100, and different choices of Ap. Continuous lines represent the theoretically obtained YMMSE, while "+" show the YMMSE averaged over 50 instances obtained from the VAMP algorithm. The plot on the right shows two models with right rotationally invariant design matrices (Ap = 0 and Ap = 0.9Ip and on the left we see two models with block right rot… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

67 extracted references · 63 canonical work pages

  1. [7]

    Barbier, N

    J. Barbier, N. Macris, M. Dia, and F. Krzakala. Mutual information and optimality of approximate message-passing in random linear estimation. IEEE Transactions on Information Theory , 66(7):4270–4303, 2020

    work page 2020

  2. [4]

    Barbier, F

    J. Barbier, F. Krzakala, N. Macris, L. Miolane, and L. Zdeborová. Optimal errors and phase transitions in high-dimensional generalized linear models. Proceedings of the National Academy of Sciences , 116(12):5451–5460, 2019

    work page 2019

  3. [5]

    Barbier and N

    J. Barbier and N. Macris. The adaptive interpolation method: a simple scheme to prove replica formulas in bayesian inference. Probability theory and related fields , 174:1133–1185, 2019

    work page 2019

  4. [1]

    Ba ´nbura, D

    M. Ba ´nbura, D. Giannone, and L. Reichlin. Large bayesian vector auto regressions. Journal of applied Econometrics , 25(1):71–92, 2010

    work page 2010

  5. [2]

    Bar-Joseph, A

    Z. Bar-Joseph, A. Gitter, and I. Simon. Studying and modelling dynamic biological processes using time-series gene expression data. Nature Reviews Genetics , 13(8):552–564, 2012

    work page 2012

  6. [3]

    Barbier, M

    J. Barbier, M. Dia, N. Macris, and F. Krzakala. The mutual information in random linear estimation. In 2016 54th Annual Allerton Conference on Communication, Control, and Computing (Allerton) , pages 625–632. IEEE, 2016

    work page 2016

  7. [6]

    Barbier and N

    J. Barbier and N. Macris. The adaptive interpolation method for proving replica formulas. applications to the curie–weiss and wigner spike models. Journal of Physics A: Mathematical and Theoretical , 52(29):294002, 2019

    work page 2019

  8. [8]

    Barbier, N

    J. Barbier, N. Macris, A. Maillard, and F. Krzakala. The mutual information in random linear estimation beyond i.i.d. matrices. In 2018 IEEE International Symposium on Information Theory (ISIT) , pages 1390–1394, 2018

    work page 2018

Show all 67 references
  1. [9]

    S. Basu, S. Das, G. Michailidis, and A. Purnanandam. A high-dimensional approach to measure connectivity in the financial sector. The Annals of Applied Statistics , 18(2):922–945, 2024

    2024

  2. [10]

    Basu and D

    S. Basu and D. S. Matteson. A survey of estimation methods for sparse high-dimensional time series models. arXiv preprint arXiv:2107.14754, 2021

    2021 arXiv

  3. [11]

    Basu and G

    S. Basu and G. Michailidis. Regularized estimation in sparse high-dimensional time series models. The Annals of Statistics, 43:1535–1567, 08 2015

    2015

  4. [12]

    Bayati and A

    M. Bayati and A. Montanari. The dynamics of message passing on dense graphs, with applications to compressed sensing. IEEE Transactions on Information Theory , 57(2):764–785, 2011

    2011

  5. [13]

    Bayati and A

    M. Bayati and A. Montanari. The lasso risk for gaussian matrices. IEEE Transactions on Information Theory , 58(4):1997–2017, 2012

    1997

  6. [14]

    D. Bean, P. J. Bickel, N. E. Karoui, and B. Yu. Optimal m-estimation in high-dimensional regression. Proceedings of the National Academy of Sciences, 110(36):14563–14568, 2013

    2013

  7. [15]

    Boucheron, G

    S. Boucheron, G. Lugosi, and O. Bousquet. Concentration Inequalities, pages 208–240. Springer Berlin Heidelberg, Berlin, Heidelberg, 2004

    2004

  8. [16]

    Cakmak, O

    B. Cakmak, O. Winther, and B. H. Fleury. S-amp: Approximate message passing for general matrix ensembles. In 2014 IEEE Information Theory Workshop (ITW 2014) , pages 192–196. IEEE, 2014

    2014

  9. [17]

    E. J. Candès and P. Sur. The phase transition for the existence of the maximum likelihood estimate in high-dimensional logistic regression. The Annals of Statistics , 48(1):27 – 42, 2020

    2020

  10. [18]

    X. Chen, M. Xu, and W. B. Wu. Covariance and precision matrix estimation for high-dimensional time series. 2013

    2013

  11. [19]

    R. A. Davis, P. Zang, and T. Zheng. Sparse vector autoregressive modeling. Journal of Computational and Graphical Statistics, 25(4):1077– 1096, 2016

    2016

  12. [20]

    Donoho and A

    D. Donoho and A. Montanari. High dimensional robust m-estimation: Asymptotic variance via approximate message passing. Probability Theory and Related Fields , 166:935–969, 2016

    2016

  13. [21]

    D. L. Donoho, A. Maleki, and A. Montanari. Message-passing algorithms for compressed sensing. Proceedings of the National Academy of Sciences, 106(45):18914–18919, 2009

    2009

  14. [22]

    Dudeja and M

    R. Dudeja and M. Bakhshizadeh. Universality of linearized message passing for phase retrieval with structured sensing matrices. IEEE Transactions on Information Theory , 68(11):7545–7574, 2022

    2022

  15. [23]

    Dudeja, Y

    R. Dudeja, Y . M. Lu, and S. Sen. Universality of approximate message passing with semirandom matrices. The Annals of Probability , 51(5):1616–1683, 2023

    2023

  16. [24]

    Dudeja, S

    R. Dudeja, S. Sen, and Y . M. Lu. Spectral universality in regularized linear regression with nearly deterministic sensing matrices. IEEE Transactions on Information Theory , 2024

    2024

  17. [25]

    El Karoui

    N. El Karoui. On the impact of predictor geometry on the performance on high-dimensional ridge-regularized generalized robust regression estimators. Probability Theory and Related Fields , 70:95–175, 2018

    2018

  18. [26]

    Fletcher, M

    A. Fletcher, M. Sahraee-Ardakan, S. Rangan, and P. Schniter. Expectation consistent approximate inference: Generalizations and convergence. In 2016 IEEE International Symposium on Information Theory (ISIT) , pages 190–194. IEEE, 2016

    2016

  19. [27]

    Grenander and G

    U. Grenander and G. Szegö. Toeplitz Forms and Their Applications . University of California Press

  20. [28]

    D. Guo, S. Shamai, and S. Verdú. Mutual information and minimum mean-square error in gaussian channels. IEEE transactions on information theory, 51(4):1261–1282, 2005

    2005

  21. [29]

    M. Kac, W. W. Murdock, and G. Szegö. On the eigenvalues of certain hermitian forms. Indiana University Mathematics Journal , 2:767–800, 1953

    1953

  22. [30]

    N. E. Karoui, D. Bean, P. J. Bickel, C. Lim, and B. Yu. On robust regression with high-dimensional predictors. Proceedings of the National Academy of Sciences , 110(36):14557–14562, 2013

    2013

  23. [31]

    Krzakala, M

    F. Krzakala, M. Mézard, F. Sausset, Y . Sun, and L. Zdeborová. Statistical-physics-based reconstruction in compressed sensing. Physical Review X, 2(2):021005, 2012

    2012

  24. [32]

    Lahiry and P

    S. Lahiry and P. Sur. Universality in block dependent linear models with applications to nonlinear regression. IEEE Transactions on Information Theory, 70(12):8975–9000, 2024

    2024

  25. [33]

    Lelarge and L

    M. Lelarge and L. Miolane. Fundamental limits of symmetric low-rank matrix estimation. Probability Theory and Related Fields , 173:859–929, 2017

    2017

  26. [34]

    Li and W

    J. Li and W. Chen. Forecasting macroeconomic time series: Lasso-based approaches and their forecast combinations with dynamic factor models. International Journal of Forecasting , 30(4):996–1015, 2014

    2014

  27. [35]

    Y . Li, Z. Fan, S. Sen, and Y . Wu. Random linear estimation with rotationally-invariant designs: Asymptotics at high temperature. IEEE Transactions on Information Theory , 2023

    2023

  28. [36]

    Ma and L

    J. Ma and L. Ping. Orthogonal amp for compressed sensing with unitarily-invariant matrices. In 2016 IEEE information theory workshop (ITW), pages 280–284. IEEE, 2016

    2016

  29. [37]

    Ma and L

    J. Ma and L. Ping. Orthogonal amp. IEEE Access, 5:2020–2033, 2017. 21

    2020

  30. [38]

    M. C. Medeiros and E. F. Mendes. l1-regularization of high-dimensional time-series models with non-gaussian and heteroskedastic errors. Journal of Econometrics , 191(1):255–271, 2016

    2016

  31. [39]

    Mezard and A

    M. Mezard and A. Montanari. Information, physics, and computation . Oxford University Press, 2009

    2009

  32. [40]

    Mezard, G

    M. Mezard, G. Parisi, and M. Virasoro. Spin Glass Theory and Beyond . WORLD SCIENTIFIC, 1986

    1986

  33. [41]

    T. P. Minka. Expectation propagation for approximate bayesian inference. arXiv preprint arXiv:1301.2294 , 2013

    2013 arXiv

  34. [42]

    Montanari and B

    A. Montanari and B. N. Saeed. Universality of empirical risk minimization. In P.-L. Loh and M. Raginsky, editors, Proceedings of Thirty Fifth Conference on Learning Theory , volume 178 of Proceedings of Machine Learning Research , pages 4310–4312. PMLR, 02–05 Jul 2022

    2022

  35. [43]

    Nardi and A

    Y . Nardi and A. Rinaldo. Autoregressive process modeling via the lasso procedure. Journal of Multivariate Analysis , 102(3):528–549, 2011

    2011

  36. [44]

    Nishimori

    H. Nishimori. Statistical Physics of Spin Glasses and Information Processing: an Introduction . Oxford University Press, Oxford; New York, 2001

    2001

  37. [45]

    Oymak, C

    S. Oymak, C. Thrampoulidis, and B. Hassibi. The squared-error of generalized lasso: A precise analysis. In 2013 51st Annual Allerton Conference on Communication, Control, and Computing (Allerton) , pages 1002–1009, 2013

    2013

  38. [46]

    Potters and J.-P

    M. Potters and J.-P. Bouchaud. A first course in random matrix theory: for physicists, engineers and data scientists . Cambridge University Press, 2020

    2020

  39. [47]

    S. Rangan. Generalized approximate message passing for estimation with random linear mixing. In 2011 IEEE International Symposium on Information Theory Proceedings , pages 2168–2172. IEEE, 2011

    2011

  40. [48]

    Rangan, P

    S. Rangan, P. Schniter, and A. K. Fletcher. Vector approximate message passing. IEEE Transactions on Information Theory , 65(10):6664– 6684, 2019

    2019

  41. [49]

    Reeves and H

    G. Reeves and H. D. Pfister. The replica-symmetric prediction for compressed sensing with gaussian matrices is exact. In 2016 IEEE International Symposium on Information Theory (ISIT) , pages 665–669, 2016

    2016

  42. [50]

    Sporns, J

    O. Sporns, J. Faskowitz, A. S. Teixeira, S. A. Cutts, and R. F. Betzel. Dynamic expression of brain functional systems disclosed by fine-scale analysis of edge time series. Network Neuroscience, 5(2):405–433, 2021

    2021

  43. [51]

    M. Stojnic. A framework to characterize performance of lasso algorithms. arXiv preprint arXiv:1303.7291 , 2013

    2013 arXiv

  44. [52]

    M. Stojnic. Meshes that trap random subspaces. arXiv preprint arXiv:1304.0003 , 2013

    2013 arXiv

  45. [53]

    M. Stojnic. Upper-bounding ℓ1-optimization weak thresholds. arXiv preprint arXiv:1303.7289 , 2013

    2013 arXiv

  46. [54]

    Sur and E

    P. Sur and E. J. Candès. A modern maximum-likelihood theory for high-dimensional logistic regression. Proceedings of the National Academy of Sciences , 116(29):14516–14525, 2019

    2019

  47. [55]

    P. Sur, Y . Chen, and E. J. Candes. The likelihood ratio test in high-dimensional logistic regression is asymptotically a rescaled chi-square. Probability Theory and Related Fields , 175:487–558, 2019

    2019

  48. [56]

    Takeda, S

    K. Takeda, S. Uda, and Y . Kabashima. Analysis of cdma systems that are characterized by eigenvalue spectrum. Europhysics Letters, 76(6):1193, nov 2006

    2006

  49. [57]

    T. Tanaka. A statistical-mechanics approach to large-system analysis of cdma multiuser detectors. IEEE Transactions on Information Theory, 48(11):2888–2910, 2002

    2002

  50. [58]

    Thrampoulidis, E

    C. Thrampoulidis, E. Abbasi, and B. Hassibi. Precise error analysis of regularized m -estimators in high dimensions. IEEE Transactions on Information Theory , 64(8):5592–5628, 2018

    2018

  51. [59]

    Thrampoulidis, S

    C. Thrampoulidis, S. Oymak, and B. Hassibi. Regularized linear regression: A precise analysis of the estimation error. In P. Grünwald, E. Hazan, and S. Kale, editors, Proceedings of The 28th Conference on Learning Theory , volume 40 of Proceedings of Machine Learning Research,...

    2015

  52. [60]

    W. F. Trench. Spectral distribution of generalized Kac-Murdock-Szegö matrices. Linear Algebra and its Applications , 347(1):251–273, 2002

    2002

  53. [61]

    A. M. Tulino, G. Caire, S. Verdú, and S. Shamai. Support recovery with sparsely sampled free random matrices. IEEE Transactions on Information Theory, 59(7):4243–4271, 2013

    2013

  54. [62]

    A. M. Tulino and S. Verdú. Random matrix theory and wireless communications. Foundations and Trends in Communications and Information Theory, 1(1):1–182, 2004

    2004

  55. [63]

    S. Wang, H. Weng, and A. Maleki. Which bridge estimator is the best for variable selection? The Annals of Statistics , 48(5):2791 – 2823, 2020

    2020

  56. [64]

    K. C. Wong, Z. Li, and A. Tewari. Lasso guarantees for β-mixing heavy-tailed time series. The Annals of Statistics , 48(2):1124–1142, 2020

    2020

  57. [65]

    Wu and Y

    W.-B. Wu and Y . N. Wu. Performance bounds for parameter estimates of high-dimensional linear models with correlated errors. 2016

    2016

  58. [66]

    Zdeborová and F

    L. Zdeborová and F. Krzakala. Statistical physics of inference: thresholds and algorithms. Advances in Physics , 65(5):453–552, 2016

    2016

  59. [67]

    Q. Zhao, P. Sur, and E. J. Candès. The asymptotic distribution of the MLE in high-dimensional logistic models: Arbitrary covariance. Bernoulli, 28(3):1835 – 1861, 2022. APPENDIX A REPLICA CALCULATION In this appendix we derive the replica symmetric potential (A.46) of the free...

    2022

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.