Pith. sign in

REVIEW 2 major objections 4 minor 39 references

Computational Thresholds in Multi-Modal Learning via the Spiked Matrix-Tensor Model

T0 review · 2 major / 4 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A shared latent signal lets an efficient algorithm recover matrix and tensor at thresholds that joint empirical risk minimization provably worsens, while a matrix-first curriculum restores the optimal thresholds.

desk verdict Solid new model and clean thresholds for Bayes-AMP and sequential spectral methods, but the ERM-degradation claim overstates what ML-AMP fixed-point analysis can prove. read the letter →

arxiv 2506.02664 v1 pith:UNO6YWV5 submitted 2025-06-03 stat.ML cond-mat.dis-nncs.LG

Hugo Tabanelli , Pierre Mergny , Lenka Zdeborova , Florent Krzakala This is my paper
classification stat.MLcond-mat.dis-nncs.LG MSC 60B2062H2582B44
keywords spikedmatrix-tensormodelBayes-AMPstateevolutionweakrecoverythresholdphasetransitioncurriculumlearningmulti-modaltensorPCA
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces a non-symmetric spiked matrix-tensor model in which a noisy matrix and a noisy third-order tensor are correlated because they share one latent vector $u^\star$: the matrix carries $Y_m = \sqrt{\Delta_m}Z + u^\star(v^\star)^\top/\sqrt{N_1}$ and the tensor carries $Y_t = \sqrt{\Delta_t}Z + u^\star \otimes x^\star \otimes y^\star/N_1$. It aims to establish that this correlation changes what is computationally easy: a Bayes-AMP algorithm, whose updates are posterior means, recovers the matrix and then the tensor at $O(1)$ noise thresholds, whereas an isolated tensor spike is efficiently recoverable only at noise of order $O(N^{-1/2})$. For the more practical empirical-risk-minimization setting, studied through an ML-AMP proxy for gradient descent, the paper claims that joint fitting is self-defeating: the tensor term effectively shrinks the matrix aspect ratio from $\alpha_2$ to $\tilde{\alpha}_2 \leq \alpha_2$, so the matrix threshold moves from $\sqrt{\alpha_2}$ to the strictly worse $\sqrt{\tilde{\alpha}_2}$. Finally, it claims that a sequential curriculum---matrix PCA first, then contracted tensor PCA---restores the optimal Bayesian thresholds and can be implemented by two spectral decompositions. If true, the results give a precise, solvable illustration of why the order in which modalities are learned can matter as much as the information they contain.

What carries the argument

The central object is state evolution for two variants of Approximate Message Passing. For the Bayes denoiser $\eta(a,b)=b/(1+a)$, the empirical overlaps follow the low-dimensional recurrence $q^{t+1} = f^{\mathrm{Bayes}}(g_{\rho=1}(q^t))$ with $g_1 = \alpha_2 q_2/\Delta_m + \alpha_3\alpha_4 q_3q_4/\Delta_t$; for the maximum-likelihood denoiser, which projects to the sphere, the analogous ML state evolution carries the effective ratio $\tilde{\alpha}_2$. Stability analysis of the zero fixed point, the matrix-only fixed point, and the matrix-tensor fixed point of these recurrences produces all thresholds. The sequential result is carried by the BBP phase transition: after PCA on the matrix, the contracted tensor is again a rank-one spiked matrix, so the second spectral step has closed-form overlaps and inherits the original matrix threshold.

What would settle it

Fix $\rho=1$ and choose parameters with $\sqrt{\tilde{\alpha}_2} < \Delta_m < \sqrt{\alpha_2}$, where joint ML predicts no recovery but the Bayes and matrix-first analyses predict recovery. Run spherical gradient descent on the joint ML loss from a matrix-SVD-informed initialization: if it systematically recovers the matrix, the ML-AMP proxy under-predicts empirical risk minimization, and if it fails from an uninformative initialization where the ML state evolution says the matrix fixed point is stable, the proxy over-predicts.

Watch

Extended reading notes

Core claim

The paper studies weak recovery in the non-symmetric spiked matrix-tensor model with a shared spike, and its central claim is a complete phase diagram for three algorithms. Bayes-AMP recovers the matrix exactly when $\Delta_m < \sqrt{\alpha_2}$ and recovers the tensor as well as soon as $\Delta_t < \delta_c^{\mathrm{Bayes}} = \sqrt{\alpha_3 \alpha_4}\,\frac{\alpha_2 - \Delta_m^2}{\alpha_2 + \Delta_m}$, both $O(1)$ noise thresholds, whereas an isolated tensor of this form is efficiently recoverable only at noise of order $O(N^{-1/2})$. The correlation thus induces a staircase phenomenon: learning $u^\star$ from the matrix jump-starts learning $x^\star$ and $y^\star$ from the tensor, and the tensor cannot be recovered independently of the matrix. Joint ML-AMP, the paper's proxy for empirical risk minimization by gradient descent, replaces $\alpha_2$ by the strictly smaller effective aspect ratio $\tilde{\alpha}_2 = \alpha_2 \frac{\alpha_2/\Delta_m}{\alpha_2/\Delta_m + \alpha_3\alpha_4/(\rho^2 \Delta_t)}$, so the matrix threshold moves from $\sqrt{\alpha_2}$ to $\sqrt{\tilde{\alpha}_2}$ and the tensor-recovery region shrinks: adding the tensor modality makes the otherwise easy matrix task harder, and no finite weight $\rho$ repairs the loss. Estimating the matrix first by PCA and then applying PCA to the contracted tensor $Y_t \cdot \hat{u}/\sqrt{N_4}$ restores the Bayesian thresholds and produces closed-form overlaps for all four signal components.

Load-bearing premise

The claim that empirical risk minimization fails depends on the unproven assumption that ML-AMP trajectories, basins of attraction, and state evolution faithfully represent spherical gradient descent, since only the fixed points of the two dynamics are shown to coincide.

Editorial extensions

If this is right

  • A pure tensor spike that is computationally hard at moderate noise becomes efficiently recoverable whenever it is statistically correlated with a matrix spike through one shared factor.
  • Joint empirical risk minimization over heterogeneous losses can be strictly worse than Bayesian optimal inference: the tensor term degrades the matrix threshold, and no finite loss weight fully repairs it.
  • Learning order is a computational resource: a matrix-first sequential curriculum attains the optimal Bayesian thresholds while simultaneous optimization does not.
  • The sequential scheme is implementable with off-the-shelf PCA, so the threshold gain is available to a simple spectral pipeline.
  • The phase diagrams contain hard phases and spinodal transitions where an informative initialization, such as a matrix SVD warm start, recovers signals that an uninformative initialization cannot reach.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editor's extension: if ML-AMP faithfully mirrors gradient descent, the model predicts that in practical multi-modal training a shared representation trained jointly with a high-loss tensor branch can be degraded relative to training it with the easier modality alone, making curricula or staged fine-tuning the safer design.
  • Editor's extension: the staircase mechanism is stated for one shared vector and one tensor channel; the same state-evolution analysis would likely extend to multiple shared factors or higher-order tensor channels, producing a sequence of thresholds in which each learned factor lowers the noise needed for the next.
  • Editor's extension: because the pure tensor problem has a large statistical-to-computational gap, the $O(1)$ tensor threshold here predicts a strong finite-size benefit; a direct finite-$N$ comparison of Bayes-AMP and joint gradient descent across that gap would sharpen the practical reading.
  • Editor's extension: a falsifiable design rule follows: warm-starting tensor learning with a matrix-PCA estimate should beat joint training even when both use identical information, and performance should degrade monotonically as the tensor weight in the loss grows.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper studies a non-symmetric spiked matrix-tensor model in which the matrix spike and the tensor spike share the same latent vector u*. It analyzes three algorithmic settings: Bayes-AMP, a maximum-likelihood AMP variant used as a proxy for empirical risk minimization (ERM) via gradient descent, and a sequential spectral method that first estimates the matrix and then the tensor. The main claims are closed-form weak-recovery thresholds: for Bayes-AMP, matrix recovery at Δ_m = sqrt(α_2) and joint matrix-tensor recovery at Δ_t = sqrt(α_3 α_4)(α_2 − Δ_m^2)/(α_2 + Δ_m); for joint ML-AMP, the matrix threshold is strictly worse, Δ_m = sqrt(α~_2) with α~_2 = α_2 (α_2/Δ_m)/(α_2/Δ_m + α_3 α_4/(ρ^2 Δ_t)); and for sequential spectral estimation, the Bayes-optimal thresholds are recovered. The theoretical state-evolution predictions are compared with finite-size simulations for the sequential spectral method, Bayes-AMP, and ML-AMP.

Significance. If the results hold, this is a valuable contribution to the theory of multi-modal high-dimensional inference. It provides an exactly solvable model in which structural correlation between a hard tensor channel and an easy matrix channel creates a staircase effect, making tensor recovery possible at O(1) noise levels and yielding a simple curriculum-learning strategy with optimal thresholds. The main positive features are the explicit, parameter-free threshold formulas; the clear separation of the Bayesian, ML-AMP, and sequential regimes; and the finite-size simulations in Figures 3, 8, and 9 that match the theoretical SE curves. The conceptual message that learning order matters is interesting and well illustrated. However, the ERM-degradation claim is currently established only for ML-AMP, not for actual gradient descent, and the Bayes-AMP fixed-point analysis relies on an imported free-energy ansatz; these points need to be addressed before the paper's headline claims can be accepted.

major comments (2)
  1. [Theorem 3.1 and Appendix E] The proof of Theorem 3.1(iii) uses a free-energy potential Φ~(q) in Appendix E that the authors state is "adapted" from the non-symmetric tensor model [13] and the symmetric matrix-tensor model [24], rather than derived for the non-symmetric model considered here. Because this free energy is used to assert that an all-nonzero stable fixed point exists and is selected in R_t, the sharp phase diagram of Theorem 3.1 is conditional on an unproven ansatz. Please either prove the free-energy expression (or its needed properties) for this model, or replace the free-energy selection argument with a direct stability and basin analysis, and state explicitly in the main text which parts of Theorem 3.1 are rigorous.
  2. [Definition 2.3] The weak-recovery definition appears dimensionally inconsistent. With estimators normalized so that ||w_k|| = Θ(√N_k), the quantity ||(u ⊗ v)^T (w_1 ⊗ w_2)|| equals |<u,w_1>| |<v,w_2>|, which is Θ(N_1 N_2) for estimators positively correlated with the spikes, not Θ(1) as written. As stated, neither the weak-recovery nor the no-recovery conditions can be satisfied in the asymptotic regime considered. The definition should rescale the overlap (for example by dividing by N_1 N_2 or by using normalized unit-norm vectors) so that the stated Θ(1) conditions match the mean-field overlaps q_k used throughout the paper.
minor comments (4)
  1. [Introduction] There is a typo in the third contribution bullet: "in constrast" should be "in contrast."
  2. [Appendix B] The paragraph beginning "This result naturally arises by replacing u_t with u_mat..." is repeated almost verbatim two paragraphs later; one copy should be deleted.
  3. [Section 5] The statement that the sequential method "recovers the optimal Bayesian thresholds" is correct, but the wording could be sharpened: the sequential spectral method achieves the same threshold values as Bayes-AMP, but its overlaps (q_s in Eqs. 5.1–5.2) are not identical to the Bayes-AMP overlaps, so the claim concerns thresholds, not per-coordinate estimation performance.
  4. [Appendix A.2] The ML-SE equations in Appendix A.2 are said to follow from [28], but the non-separable denoiser η_ML requires the augmented divergence definition introduced in Definition 2.2; it would be helpful to state exactly how the standard SE result is extended to this non-separable case, since this is not a direct application of the usual theorem.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular derivation: the thresholds are explicit SE stability conditions and BBP spectral limits, with self-citations present but not load-bearing; the Sec. 4 ERM/ML-AMP bridge is an extrapolation, not a circular reduction.

full rationale

The central thresholds are derived from the paper's own state-evolution fixed-point and stability analyses, not from fitted constants. Theorem 3.1 and App. D obtain the Bayes-AMP matrix threshold Delta_m = sqrt(alpha_2) and the tensor threshold Delta_t = sqrt(alpha_3 alpha_4) (alpha_2 - Delta_m^2)/(alpha_2 + Delta_m) by linearizing the Bayes-SE equations around the trivial and matrix fixed points; the overlaps follow from the closed-form solution in Lemma C.1. The ML-AMP thresholds in Theorem 4.1 are the analogous stability conditions of the ML-SE map, giving the effective alpha_tilde_2 boundary, and the claimed degradation is the algebraic consequence alpha_tilde_2 <= alpha_2. Theorem 5.1 is derived independently by applying the BBP lemma to the spiked matrix and then to the contracted tensor, with matching finite-size simulations in Fig. 3 that are not fitting exercises. The self-citations that appear, such as [24] for the symmetric matrix-tensor free energy and [36] for the Bayes-AMP optimality conjecture, are external published results or are backed by the independent work [18]; they do not constitute the derivation of the thresholds. The one flagged weakness is a correctness gap rather than circularity: Section 4 explicitly says it studies ML-AMP because it shares fixed points with gradient descent, while Proposition 4.1 proves only fixed-point equality, not equality of state-evolution trajectories or attraction basins; the abstract's statement that empirical risk minimization for joint learning fails therefore extrapolates beyond the proven claim. This does not reduce any equation to its inputs, so it does not raise the circularity score. The score of 2 reflects the presence of minor self-citations that are not load-bearing for the main closed-form results.

Assumptions & free parameters 1 free parameters · 6 assumptions · 0 invented entities

The paper introduces no fitted constants and no new physical entities. Its contributions are algebraic derivations built on standard AMP, state evolution, random matrix theory, and free energy methods. The main load-bearing assumptions are the imported state evolution theorem, the optimality conjecture for Bayes-AMP, and the ML-AMP-to-GD proxy.

free parameters (1)
  • rho (relative tensor weight in the ML loss) = 1 in the ML loss; rho going to infinity corresponds to the sequential limit
    Introduced in Section 4 to control the relative weight between matrix and tensor losses. It enters the ML-AMP threshold alpha_tilde_2 and the phase boundaries, but it is a modeling knob chosen by hand, not a constant fitted to data.
assumptions (6)
  • domain assumption Gaussian i.i.d. priors and noise with high-dimensional limits at fixed aspect ratios
    The model and all theorems assume this asymptotic setting and do not analyze finite-size or non-Gaussian behavior.
  • standard math State evolution correctly characterizes the AMP family (Proposition 2.1)
    Imported from Javanmard-Montanari and Feng et al.; no proof is given here, and the non-separable ML denoiser requires a broadening of the standard framework.
  • domain assumption Bayes-AMP is optimal among first-order algorithms
    Used to describe the Bayes-AMP thresholds as computational thresholds; relies on [18] and conjectures in [36], not proved for this model.
  • domain assumption ML-AMP is a faithful proxy for gradient descent and empirical risk minimization
    Section 4 and Proposition 4.1 establish only fixed-point equivalence; the dynamics are not shown to match, but the ERM conclusions depend on this.
  • standard math Free energy variational formula for the non-symmetric model
    Appendix E adapts the expression from the non-symmetric tensor model [13] and the symmetric matrix-tensor model [24] without re-deriving it.
  • standard math BBP phase transition for rectangular spiked matrices (Lemma 5.1)
    Stated and applied without proof in Theorem 5.1; it is a standard random matrix theory result.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Computational Thresholds in Multi-Modal Learning via the Spiked Matrix-Tensor Model." pith.science (2026). https://pith.science/paper/UNO6YWV5

@misc{pith2026250602664,
  author       = {Pith},
  title        = {Pith review of: Computational Thresholds in Multi-Modal Learning via the Spiked Matrix-Tensor Model},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UNO6YWV5}},
  note         = {Machine review of arXiv:2506.02664}
}
read the original abstract

We study the recovery of multiple high-dimensional signals from two noisy, correlated modalities: a spiked matrix and a spiked tensor sharing a common low-rank structure. This setting generalizes classical spiked matrix and tensor models, unveiling intricate interactions between inference channels and surprising algorithmic behaviors. Notably, while the spiked tensor model is typically intractable at low signal-to-noise ratios, its correlation with the matrix enables efficient recovery via Bayesian Approximate Message Passing, inducing staircase-like phase transitions reminiscent of neural network phenomena. In contrast, empirical risk minimization for joint learning fails: the tensor component obstructs effective matrix recovery, and joint optimization significantly degrades performance, highlighting the limitations of naive multi-modal learning. We show that a simple Sequential Curriculum Learning strategy-first recovering the matrix, then leveraging it to guide tensor recovery-resolves this bottleneck and achieves optimal weak recovery thresholds. This strategy, implementable with spectral methods, emphasizes the critical role of structural correlation and learning order in multi-modal high-dimensional inference.

Figures

Figures reproduced from arXiv: 2506.02664 by the authors.

Figure 1
Figure 1. Phase diagrams for Bayes-AMP algorithm, (a) in [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗
Figure 2
Figure 2. Phase diagrams for ML-AMP algorithm in (∆m, ∆t) space (a) with size ratios α2 = 1.5, α3 = 0.8, and α4 = 1 and in (α2, α3) space (b) with noise ∆m = 0.7, ∆t = 0.8 and a size ratio α4 = 1. The purple and green dashed￾dotted lines indicate the phase transitions between the regions R˜ 0 and R˜m, and between R˜m and R˜ t, respectively, for several values of the parameter ρ. These transitions are implicitly defined by ∆m … view at source ↗
Figure 3
Figure 3. Empirical and theoretical MSE vs. ∆m with parameters ∆t = 0.3, N1 = 103 , N2 = 1.5 . 103 , N3 = 0.8 . 103 , N4 = 103 . The empirical values (dots) are computed numerically via the the sequential spectral method, averaged over 500 samples. The error bars are not visible on the plots due to their small magnitude. Their theoretical predictions (solid lines) are given by the overlaps (q s 1 , qs 2 , qs 3 , qs 4 ) define… view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Factor graph of the posterior distribution. The displayed couplings are not exhaustive; blank factors illustrate [PITH_FULL_IMAGE:figures/full_fig_p014_4.png]
Figure 5
Figure 5. Figure 5: Mean Square Error (MSE) as a function of [PITH_FULL_IMAGE:figures/full_fig_p024_5.png]
Figure 6
Figure 6. Figure 6: Phase diagrams for Bayes-AMP algorithm, (a) in [PITH_FULL_IMAGE:figures/full_fig_p025_6.png]
Figure 7
Figure 7. Figure 7: Mean Square Error (MSE) as a function of [PITH_FULL_IMAGE:figures/full_fig_p026_7.png]
Figure 8
Figure 8. Figure 8: Empirical and theoretical MSE vs. ∆m in the Bayesian case, with parameters ∆t = 0.6, N1 = 103 , N2 = 1.5 . 103 , N3 = 0.8 . 103 , N4 = 103 . The empirical values (dots) are computed numerically via Bayes-AMP algorithm initialized with an SVD on the matrix, averaged ove…
Figure 9
Figure 9. Figure 9: Empirical and theoretical MSE vs. ∆m in the ML case, with parameters ∆t = 0.6, N1 = 103 , N2 = 1.5 . 103 , N3 = 0.8 . 103 , N4 = 103 , and ρ = 1. The empirical values (dots) are computed numerically via ML-AMP algorithm initialized with an SVD on the matrix, averaged o…

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

39 extracted references · 31 canonical work pages

  1. [13]

    The layered structure of tensor estimation and its mutual information

    Jean Barbier, Nicolas Macris, and Léo Miolane. The layered structure of tensor estimation and its mutual information. In 2017 55th Annual Allerton Conference on Communication, Control, and Computing (Allerton), pages 1056–1063. IEEE, 2017

    work page 2017

  2. [24]

    Marvels and Pitfalls of the Langevin Algorithm in Noisy High-Dimensional Inference

    Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, Pierfrancesco Urbani, and Lenka Zdeborová. Marvels and Pitfalls of the Langevin Algorithm in Noisy High-Dimensional Inference. Physical Review X, 10(1), March 2020

    work page 2020

  3. [1]

    Deep learning

    Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. nature, 521(7553):436–444, 2015

    2015

  4. [2]

    Multimodal deep learning

    Jiquan Ngiam, Aditya Khosla, Mingyu Kim, Juhan Nam, Honglak Lee, Andrew Y Ng, et al. Multimodal deep learning. In ICML, volume 11, pages 689–696, 2011

    work page 2011

  5. [3]

    Multimodal machine learning: A survey and taxonomy

    Tadas Baltrušaitis, Chaitanya Ahuja, and Louis-Philippe Morency. Multimodal machine learning: A survey and taxonomy. IEEE transactions on pattern analysis and machine intelligence, 41(2):423–443, 2018

    2018

  6. [4]

    Curriculum learning

    Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. Curriculum learning. In Proceedings of the 26th annual international conference on machine learning, pages 41–48, 2009

    2009

  7. [5]

    Ben Arous, J

    G. Ben Arous, J. Baik, and S. Péché. Phase transition of the largest eigenvalue for non-null complex sample covariance matrices. Annals of probability: An official journal of the Institute of Mathematical Statistics , 33(5):1643–1697, 2005

    work page 2005

  8. [6]

    Optimal shrinkage of eigenvalues in the spiked covariance model

    David L Donoho, Matan Gavish, and Iain M Johnstone. Optimal shrinkage of eigenvalues in the spiked covariance model. Annals of statistics, 46(4):1742, 2018

    work page 2018

Show all 39 references
  1. [7]

    A statistical model for tensor PCA

    Emile Richard and Andrea Montanari. A statistical model for tensor PCA. Advances in neural information processing systems, 27, 2014

    2014

  2. [8]

    Statistical and compu- tational phase transitions in spiked tensor estimation

    Thibault Lesieur, Léo Miolane, Marc Lelarge, Florent Krzakala, and Lenka Zdeborová. Statistical and compu- tational phase transitions in spiked tensor estimation. In 2017 IEEE International Symposium on Information Theory (ISIT), pages 511–515. IEEE, 2017

    2017

  3. [9]

    Tensor principal component analysis via sum-of-square proofs

    Samuel B Hopkins, Jonathan Shi, and David Steurer. Tensor principal component analysis via sum-of-square proofs. In Conference on Learning Theory, pages 956–1006. PMLR, 2015

    2015

  4. [10]

    Wein, and Afonso S

    Amelia Perry, Alexander S. Wein, and Afonso S. Bandeira. Statistical limits of spiked tensor models. Annales de l’Institut Henri Poincaré, Probabilités et Statistiques, 56(1):230–264, February 2020. Publisher: Institut Henri Poincaré

    2020

  5. [11]

    The Kikuchi hierarchy and tensor PCA

    Alexander S Wein, Ahmed El Alaoui, and Cristopher Moore. The Kikuchi hierarchy and tensor PCA. In 2019 IEEE 60th Annual Symposium on Foundations of Computer Science (FOCS), pages 1446–1468. IEEE, 2019

    2019

  6. [12]

    Algorithmic thresholds for tensor PCA

    Gerard Ben Arous, Reza Gheissari, and Aukosh Jagannath. Algorithmic thresholds for tensor PCA. The Annals of Probability, 48(4):2052–2087, 2020. 11

    2020

  7. [14]

    Statistical mechanics of low-rank tensor decomposition

    Jonathan Kadmon and Surya Ganguli. Statistical mechanics of low-rank tensor decomposition. Advances in Neural Information Processing Systems, 31, 2018

    2018

  8. [15]

    The staircase property: How hierarchical structure can guide deep learning

    Emmanuel Abbe, Enric Boix-Adsera, Matthew S Brennan, Guy Bresler, and Dheeraj Nagaraj. The staircase property: How hierarchical structure can guide deep learning. Advances in Neural Information Processing Systems, 34:26989–27002, 2021

    2021

  9. [16]

    SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics

    Emmanuel Abbe, Enric Boix Adsera, and Theodor Misiakiewicz. SGD learning on neural networks: leap complexity and saddle-to-saddle dynamics. In The Thirty Sixth Annual Conference on Learning Theory, pages 2552–2623. PMLR, 2023

    2023

  10. [17]

    Fundamental limits of weak learnability in high-dimensional multi-index models

    Emanuele Troiani, Yatin Dandi, Leonardo Defilippis, Lenka Zdeborova, Bruno Loureiro, and Florent Krzakala. Fundamental limits of weak learnability in high-dimensional multi-index models. In International Conference on Machine Learning, June 2024

    2024

  11. [18]

    The high-dimensional asymptotics of first order methods with random data

    Michael Celentano, Chen Cheng, and Andrea Montanari. The high-dimensional asymptotics of first order methods with random data. arXiv preprint arXiv:2112.07572, 2021

    2021 arXiv

  12. [19]

    Information-theoretically optimal sparse PCA

    Yash Deshpande and Andrea Montanari. Information-theoretically optimal sparse PCA. In 2014 IEEE Interna- tional Symposium on Information Theory, pages 2197–2201. IEEE, 2014

    2014

  13. [20]

    Phase transition of the largest eigenvalue for non-null complex sample covariance matrices

    Jinho Baik, Gérard Ben Arous, and Sandrine Péché. Phase transition of the largest eigenvalue for non-null complex sample covariance matrices. The Annals of Probability, 33(5):1643–1697, September 2005. Publisher: Institute of Mathematical Statistics

    2005

  14. [21]

    Constrained low-rank matrix estimation: phase transitions, approximate message passing and applications

    Thibault Lesieur, Florent Krzakala, and Lenka Zdeborová. Constrained low-rank matrix estimation: phase transitions, approximate message passing and applications. Journal of Statistical Mechanics: Theory and Experiment, 2017(7):073403, jul 2017

    2017

  15. [22]

    Spherical 2+p Spin-Glass model: An exactly solvable model for Glass to Spin-Glass transition

    Andrea Crisanti and Luca Leuzzi. Spherical 2+p Spin-Glass model: An exactly solvable model for Glass to Spin-Glass transition. Physical review letters, 93(21):217203, 2004

    2004

  16. [23]

    Parisi formula, disorder chaos and fluctuation for the ground state energy in the spherical mixed p-spin models

    Wei-Kuo Chen and Arnab Sen. Parisi formula, disorder chaos and fluctuation for the ground state energy in the spherical mixed p-spin models. Communications in Mathematical Physics, 350:129–173, 2017

    2017

  17. [25]

    Passed & Spurious: Descent Algorithms and Local Minima in Spiked Matrix-Tensor Models

    Stefano Sarao Mannelli, Florent Krzakala, Pierfrancesco Urbani, and Lenka Zdeborova. Passed & Spurious: Descent Algorithms and Local Minima in Spiked Matrix-Tensor Models. In Proceedings of the 36th International Conference on Machine Learning, pages 4333–4342. PMLR, May 2019....

    2019

  18. [26]

    Who is Afraid of Big Bad Minima? Analysis of gradient-flow in spiked matrix-tensor models

    Stefano Sarao Mannelli, Giulio Biroli, Chiara Cammarota, Florent Krzakala, and Lenka Zdeborová. Who is Afraid of Big Bad Minima? Analysis of gradient-flow in spiked matrix-tensor models. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019

    2019

  19. [27]

    Donoho, Arian Maleki, and Andrea Montanari

    David L. Donoho, Arian Maleki, and Andrea Montanari. Message-passing algorithms for compressed sensing. Proceedings of the National Academy of Sciences, 106(45):18914–18919, November 2009. Publisher: Proceedings of the National Academy of Sciences

    2009

  20. [28]

    State evolution for general approximate message passing algorithms, with applications to spatial coupling

    Adel Javanmard and Andrea Montanari. State evolution for general approximate message passing algorithms, with applications to spatial coupling. Information and Inference: A Journal of the IMA, 2(2):115–144, December 2013

    2013

  21. [29]

    Sampling from mean-field Gibbs measures via diffusion processes

    Ahmed El Alaoui, Andrea Montanari, and Mark Sellke. Sampling from mean-field Gibbs measures via diffusion processes. arXiv preprint arXiv:2310.08912, 2023. 12

    2023

  22. [30]

    The merged-staircase property: a necessary and nearly sufficient condition for sgd learning of sparse functions on two-layer neural networks

    Emmanuel Abbe, Enric Boix Adsera, and Theodor Misiakiewicz. The merged-staircase property: a necessary and nearly sufficient condition for sgd learning of sparse functions on two-layer neural networks. In Conference on Learning Theory, pages 4782–4887. PMLR, 2022

    2022

  23. [31]

    A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets

    Khaled Bayoudh, Raja Knani, Fayçal Hamdaoui, and Abdellatif Mtibaa. A survey on deep multimodal learning for computer vision: advances, trends, applications, and datasets. The Visual Computer, 38(8):2939–2970, 2022

    2022

  24. [32]

    Simultaneous dimensionality reduction: A data efficient approach for multimodal representations learning

    Eslam Abdelaleem, Ahmed Roman, K Michael Martini, and Ilya Nemenman. Simultaneous dimensionality reduction: A data efficient approach for multimodal representations learning. arXiv preprint arXiv:2310.04458, 2023

    2023 arXiv

  25. [33]

    Optimal thresholds and algorithms for a model of multi-modal learning in high dimensions

    Christian Keup and Lenka Zdeborová. Optimal thresholds and algorithms for a model of multi-modal learning in high dimensions. arXiv preprint arXiv:2407.03522, 2024

    2024

  26. [34]

    Multimodal data integration and cross-modal querying via orchestrated approximate message passing

    Sagnik Nandy and Zongming Ma. Multimodal data integration and cross-modal querying via orchestrated approximate message passing. arXiv preprint arXiv:2407.19030, 2024

    2024

  27. [35]

    A unifying tutorial on approximate message passing

    Oliver Y Feng, Ramji Venkataramanan, Cynthia Rush, Richard J Samworth, et al. A unifying tutorial on approximate message passing. Foundations and Trends® in Machine Learning, 15(4):335–536, 2022

    2022

  28. [36]

    Disordered systems insights on computational hardness

    David Gamarnik, Cristopher Moore, and Lenka Zdeborová. Disordered systems insights on computational hardness. Journal of Statistical Mechanics: Theory and Experiment, 2022(11):114015, 2022. 13 A Detailed expressions of the algorithms In this appendix, we provide the explicit i...

    2022

  29. [37]

    ∼ N(0, 1), and we recall [g(qt)]1 = α2 ∆m qt 2 + α3α4 ∆t qt 3qt

    Gi = [g(qt)]1 , with the gaussian random vector Gi i.i.d. ∼ N(0, 1), and we recall [g(qt)]1 = α2 ∆m qt 2 + α3α4 ∆t qt 3qt

  30. [38]

    Thus, one can write the closed-form update: qt+1 1 = Eu⋆ i ,Gi η [g(qt)]1, [g(qt)]1u⋆ i + q [g(qt)]1 Gi u⋆ i = [g(qt)]1 1 + [g(qt)]1 . Repeating this procedure for the remaining variables yields the full Bayes-SE system:    qt+1 1 = α2 ∆m qt 2+ α3 α4 ∆...

  31. [39]

    Thus, one can write the closed-form update: qt+1 1 = Eu⋆ i ,Gi h ηML [gρ(qt)]1u⋆ i + q [gρ2 (1)]1 Gi u⋆ i i = [gρ(qt)]1 q [gρ2 (1)]1 + [gρ(qt)]2 1 . Repeating this procedure for the remaining variables yields the full ML-SE system:    qt+1 1 = α2 ∆m ...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.