Finite-Iteration Local Dynamics and Warm Starts for Alternating Power Iteration in Spiked Tensor PCA

Yanjin Xiang; Zhihua Zhang

REVIEW 2 major objections 2 minor 16 references

Finite-Iteration Local Dynamics and Warm Starts for Alternating Power Iteration in Spiked Tensor PCA

T0 review · 2 major / 2 minor · reviewed 2026-06-28 · grok-4.3

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Alternating power iteration for rank-one spiked tensors enters a local basin after one sweep and then converges geometrically to a noise floor.

desk verdict The paper delivers a clean finite-iteration local error decomposition for alternating power iteration plus a generic one-sweep warm-start condition, but the high-probability noise control is only verified at the initializer. read the letter →

arxiv 2606.04065 v1 pith:FXXVIHLH submitted 2026-06-02 stat.ML cs.LGmath.STstat.TH

Yanjin Xiang , Zhihua Zhang This is my paper · ORCID

classification stat.MLcs.LGmath.STstat.TH

keywords spikedtensorPCAalternatingpoweriterationlocalconvergencewarmstartrank-onemodelmultilinearnoiseeventcentered-Graminitializationpressed-backestimate

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper establishes a finite-iteration local theory for alternating power iteration on fixed-order asymmetric rank-one spiked tensor models. Once iterates reach a small neighborhood of the planted direction, their error splits into a geometrically decaying transient term and a fixed noise floor produced by the orthogonal components of the noise tensor contracted at the planted point. A generic one-sweep argument shows that any sign-compatible initializer whose correlation gamma_N and first-sweep noise a_N satisfy a_N over gamma_N to the d-1 times omega_N,d going to zero can be made to enter an expanding but still local radius, after which the local contraction map drives the sequence to the unique informative fixed point inside that basin. The same conditions are verified for centered-Gram initialization under i.i.d. noise with finite fourth moments by leave-one-out comparisons that preserve the spike and averaged slice-contraction bounds.

What carries the argument

The one-sweep principle that turns a sign-compatible initializer with correlation gamma_N and noise level a_N satisfying a_N/(gamma_N^{d-1} omega_{N,d}) to 0 into entry inside an expanding radius r_N = o(omega_{N,d}), after which the local affine contraction map drives convergence.

What would settle it

Construct a noise distribution with finite fourth moments for which the multilinear noise event fails with non-vanishing probability yet the centered-Gram initializer still produces correlation gamma_N and first-sweep noise a_N satisfying the stated ratio condition; if the iterates then fail to enter the local basin the reduction claim is false.

Watch

Extended reading notes

Core claim

The deterministic finite-sample conditions for local convergence are stated explicitly; under a coarse fixed-order multilinear noise event they reduce to a conservative high-signal regime that works for fixed or slowly expanding local radii. After entry the local affine contraction yields convergence to the unique informative local fixed point. For centered-Gram initialization the required correlation and same-sample first-sweep noise bound hold by a signal-preserving noise-only leave-one comparison and an averaged leave-one slice-contraction estimate called a pressed-back estimate.

Load-bearing premise

The reduction of the deterministic finite-sample conditions to a high-signal regime holds only under a coarse fixed-order multilinear noise event whose probability is controlled only for i.i.d. finite-fourth-moment noise via the centered-Gram construction.

Editorial extensions

If this is right

Any initialization method that produces a sign-compatible vector with the stated correlation-to-noise ratio can be followed by a single sweep to reach the local basin.
Inside the basin the iteration converges to the informative fixed point irrespective of how entry was achieved.
The local radius may expand slowly while still guaranteeing geometric decay of the transient error term.
The leave-one comparison technique isolates the contribution of planted coordinates through ell-two weighted sums rather than worst-case incoherence.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The separation of warm-start entry from local contraction suggests that other iterative tensor methods could be analyzed by first proving a generic entry condition and then studying the local map separately.
The pressed-back estimate may extend to higher-order or symmetric tensor models where similar leave-one slice contractions can be averaged.
If the multilinear noise event can be shown to hold with high probability for a wider class of noise distributions, the local theory would apply beyond the fourth-moment setting used for the initializer.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

read the letter

The main things to know are that the local analysis separates a geometrically decaying transient from a fixed noise floor once inside a small basin, and that the warm-start argument is written so it applies to any initializer meeting observable correlation and first-sweep noise conditions.

What is new is the explicit finite-iteration decomposition that does not rely on global convergence arguments, together with the pressed-back leave-one estimate that keeps the spike fixed while averaging over deleted coordinates. The centered-Gram initializer is handled under i.i.d. finite-fourth-moment noise without worst-case incoherence bounds. These pieces look technically careful and the deterministic conditions are stated plainly.

The soft spot is the multilinear noise event. It is used to reduce the deterministic conditions to a high-signal regime, and it is controlled for the initializer, but the paper gives no uniform probability bound over the local ball or across the subsequent finite iterates. Without that, the claimed high-probability entry and convergence do not automatically follow for typical noise realizations. The reduction itself is conservative, which is fine, but the gap on the event probability is real.

This is for people working on non-convex tensor methods who already know the global analyses and want sharper local control or better initialization conditions. A reader who cares about explicit finite-sample rates in spiked tensor models will get something usable. The work is coherent on its own terms and shows honest engagement with the literature, so it deserves a serious referee even if the noise-event issue needs tightening.

Referee Report

2 major / 2 minor

Summary. The paper develops a finite-iteration local theory for simultaneous alternating power iteration on fixed-order asymmetric rank-one spiked tensor models. Once iterates enter a small neighborhood of the planted direction, the error is shown to decompose into a geometrically decaying transient plus an intrinsic noise floor arising from fixed orthogonal contractions at the planted point; the deterministic finite-sample conditions are stated explicitly and reduce to a conservative high-signal regime under a coarse fixed-order multilinear noise event. A generic one-sweep warm-start principle is separated from any particular spectral initializer: if a sign-compatible initializer satisfies a_N / (gamma_N^{d-1} omega_{N,d}) -> 0, an expanding radius r_N = o(omega_{N,d}) places the first sweep inside the local basin, after which the local affine contraction converges to the unique informative fixed point. For centered-Gram initialization the required correlation and same-sample first-sweep noise bounds are verified under i.i.d. finite-fourth-moment noise via signal-preserving leave-one comparisons and an averaged pressed-back slice-contraction estimate.

Significance. If the local decomposition and probability transfer hold, the work supplies explicit deterministic finite-sample conditions for local convergence of alternating power iteration that are independent of the particular initialization once inside the basin, together with a cleanly separated generic warm-start mechanism expressed in observable quantities. The signal-preserving leave-one comparison and pressed-back estimate for the initializer are concrete technical strengths that avoid worst-case incoherence bounds. These elements would be useful for finite-iteration analysis of tensor power methods beyond purely asymptotic regimes.

major comments (2)

[Abstract / local dynamics] Abstract and local-dynamics section: the deterministic finite-sample conditions for the geometric-transient-plus-noise-floor decomposition are stated explicitly, yet they reduce to the claimed high-signal regime only under a coarse fixed-order multilinear noise event whose probability is quantified solely for the centered-Gram initializer; no uniform bound is supplied ensuring the event holds over the local ball or across the finite number of subsequent iterates, which is load-bearing for the finite-iteration guarantee transferring to typical noise realizations with high probability.
[Warm-start principle / centered-Gram section] Warm-start and centered-Gram verification: the one-sweep principle is formulated in terms of observable gamma_N and a_N, but the pressed-back estimate and leave-one comparison are derived only for the initializer under i.i.d. finite-fourth-moment noise; the argument does not extend the same-sample control to the local iterates inside the expanding radius r_N, leaving the transfer of the high-probability guarantee incomplete.

minor comments (2)

[Abstract] Notation for omega_{N,d} and the multilinear noise event could be introduced with a short display equation or definition box on first use to improve readability for readers outside the immediate subfield.
[Abstract] The phrase 'conservative high-signal regime' is used without an explicit comparison to existing high-probability thresholds in the tensor-PCA literature; a one-sentence contrast would clarify the improvement.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the careful reading and the constructive major comments on the transfer of high-probability guarantees. We address each point below and will revise the manuscript to close the identified gaps in the uniform control of the multilinear noise event.

read point-by-point responses

Referee: [Abstract / local dynamics] Abstract and local-dynamics section: the deterministic finite-sample conditions for the geometric-transient-plus-noise-floor decomposition are stated explicitly, yet they reduce to the claimed high-signal regime only under a coarse fixed-order multilinear noise event whose probability is quantified solely for the centered-Gram initializer; no uniform bound is supplied ensuring the event holds over the local ball or across the finite number of subsequent iterates, which is load-bearing for the finite-iteration guarantee transferring to typical noise realizations with high probability.

Authors: The multilinear noise event is a fixed property of the noise tensor and is independent of any particular vector. We agree, however, that the current probability bound is derived only in the context of the centered-Gram initializer and does not yet supply an explicit uniform bound over the expanding local ball of radius r_N or over the finite subsequent iterates. Because the ball radius is o(ω_{N,d}) and the number of iterations is finite, a standard ε-net argument can be used to extend the bound while preserving the same high-probability statement. We will add this uniform control (via a covering argument) to the local-dynamics section so that the finite-iteration guarantee holds with high probability under the same noise assumptions. revision: yes
Referee: [Warm-start principle / centered-Gram section] Warm-start and centered-Gram verification: the one-sweep principle is formulated in terms of observable gamma_N and a_N, but the pressed-back estimate and leave-one comparison are derived only for the initializer under i.i.d. finite-fourth-moment noise; the argument does not extend the same-sample control to the local iterates inside the expanding radius r_N, leaving the transfer of the high-probability guarantee incomplete.

Authors: We agree that the same-sample leave-one comparison and pressed-back estimate are currently stated only for the initializer. Once the first sweep enters the local basin, the subsequent iterates remain inside the controlled radius r_N = o(ω_{N,d}) and are therefore close to the planted direction. The same leave-one structure can be applied to these iterates by replacing the initializer vector with the current local iterate and using the fact that the correlation remains bounded away from zero inside the basin; the resulting bounds differ from the initializer case only by lower-order terms that are absorbed into the existing constants. We will extend the argument in the centered-Gram section to cover the local iterates explicitly, thereby completing the high-probability transfer for the entire trajectory. revision: yes

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity; derivation self-contained with explicit conditions and verified bounds

full rationale

The paper states deterministic finite-sample conditions for the local error decomposition (geometrically decaying transient plus intrinsic noise floor) and reduces them to a high-signal regime only under an explicit coarse fixed-order multilinear noise event assumption. The warm-start analysis uses a generic one-sweep principle with observable quantities (correlation gamma_N, noise a_N) and verifies the required bounds for centered-Gram initialization via leave-one comparisons and pressed-back estimates under i.i.d. finite-fourth-moment noise; these are direct proofs, not reductions by construction or self-citation. No equations equate a claimed prediction to a fitted input, no uniqueness theorems are imported from prior self-work, and the local theory is conditioned on entry into the basin without circular dependence on the initializer construction. The derivation is therefore self-contained against its stated assumptions.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The central claims rest on i.i.d. finite-fourth-moment noise for the initializer verification and on the existence of a coarse fixed-order multilinear noise event that makes the deterministic conditions reduce to a high-signal regime. No free parameters or invented entities are introduced in the abstract.

assumptions (2)

domain assumption Noise entries are i.i.d. with finite fourth moments
Used to verify correlation and first-sweep noise bounds for centered-Gram initialization via leave-one comparison.
domain assumption A coarse fixed-order multilinear noise event holds
Allows reduction of deterministic finite-sample conditions to a conservative high-signal regime.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Finite-Iteration Local Dynamics and Warm Starts for Alternating Power Iteration in Spiked Tensor PCA." pith.science (2026). https://pith.science/paper/FXXVIHLH

@misc{pith2026260604065,
  author       = {Pith},
  title        = {Pith review of: Finite-Iteration Local Dynamics and Warm Starts for Alternating Power Iteration in Spiked Tensor PCA},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/FXXVIHLH}},
  note         = {Machine review of arXiv:2606.04065}
}

abstract

We study simultaneous alternating power iteration for fixed-order asymmetric rank-one spiked tensor models. Our main contribution is a finite-iteration local theory that is independent of any particular initialization. Once the iterates enter a sufficiently small neighborhood of the planted rank-one direction, their error decomposes into a geometrically decaying transient and an intrinsic noise floor caused by fixed orthogonal noise contractions at the planted point. The deterministic finite-sample conditions are stated explicitly, but under a coarse fixed-order multilinear noise event they reduce to a conservative high-signal regime for fixed or slowly expanding local radii. We then separate the warm-start mechanism from any specific spectral construction. A generic one-sweep principle shows that, if a sign-compatible initializer has correlation \(\gamma_N\), first-sweep noise level \(a_N\), and \(a_N/(\gamma_N^{d-1}\omega_{N,d})\to0\), then one can choose an expanding radius \(r_N=o(\omega_{N,d})\) for which the first sweep enters the local basin. After entry, the local affine contraction yields convergence to the unique informative local fixed point in that basin. For centered-Gram initialization, we verify the required correlation and same-sample first-sweep noise bound under i.i.d. finite-fourth-moment noise by a signal-preserving noise-only leave-one comparison and an averaged leave-one slice-contraction estimate, which we call a pressed-back estimate. The leave-one comparison keeps the spike fixed and averages over the deleted coordinate, so planted coordinates enter through \(\ell_2\)-weighted sums rather than worst-case incoherence bounds.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

16 extracted references · 1 canonical work pages

[1]

Kakade, and Matus Telgarsky

Animashree Anandkumar, Rong Ge, Daniel Hsu, Sham M. Kakade, and Matus Telgarsky. Tensor decompositions for learning latent variable models.Journal of Machine Learning Research, 15 (80):2773–2832, 2014

2014
[2]

Random matrices and complexity of spin glasses.Communications on Pure and Applied Mathematics, 66(2):165–201, 2013

Antonio Auffinger, Gérard Ben Arous, and Jiří Černý. Random matrices and complexity of spin glasses.Communications on Pure and Applied Mathematics, 66(2):165–201, 2013

2013
[3]

Bai, Jack W

Zhidong D. Bai, Jack W. Silverstein, and Yong Q. Yin. A note on the largest eigenvalue of a large dimensional sample covariance matrix.Journal of Multivariate Analysis, 26(2):166–168, 1988

1988
[4]

The landscape of the spiked tensor model.Communications on Pure and Applied Mathematics, 72(11):2282–2330, 2019

Gérard Ben Arous, Song Mei, Andrea Montanari, and Mihai Nica. The landscape of the spiked tensor model.Communications on Pure and Applied Mathematics, 72(11):2282–2330, 2019

2019
[5]

A multilinear singular value decomposition.SIAM Journal on Matrix Analysis and Applications, 21(4):1253–1278, 2000

Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. A multilinear singular value decomposition.SIAM Journal on Matrix Analysis and Applications, 21(4):1253–1278, 2000

2000
[6]

Hopkins, Jonathan Shi, and David Steurer

Samuel B. Hopkins, Jonathan Shi, and David Steurer. Tensor principal component analysis via sum-of-squares proofs. InProceedings of the 28th Conference on Learning Theory, volume 40 of Proceedings of Machine Learning Research, pages 956–1006, 2015

2015
[7]

Hopkins, Tselil Schramm, Jonathan Shi, and David Steurer

Samuel B. Hopkins, Tselil Schramm, Jonathan Shi, and David Steurer. Fast spectral algorithms from sum-of-squares proofs: tensor decomposition and planted sparse vectors. InProceedings of the Forty-Eighth Annual ACM Symposium on Theory of Computing, pages 178–191, 2016. 66

2016
[8]

Huang, Qing Yang, and Guang Cheng

Jiaoyang Huang, Daniel Z. Huang, Qing Yang, and Guang Cheng. Power iteration for tensor pca.Journal of Machine Learning Research, 23(128):1–47, 2022

2022

Show all 16 references

[9]

Statistical thresholds for tensor pca

Aukosh Jagannath, Patrick Lopatto, and Léo Miolane. Statistical thresholds for tensor pca. Annals of Applied Probability, 30(4):1910–1933, 2020

1910
[10]

Kolda and Brett W

Tamara G. Kolda and Brett W. Bader. Tensor decompositions and applications.SIAM Review, 51(3):455–500, 2009

2009
[11]

Singular values and eigenvalues of tensors: A variational approach

Lek-Heng Lim. Singular values and eigenvalues of tensors: A variational approach. InProceedings of the IEEE International Workshop on Computational Advances in Multi-Sensor Adaptive Processing, pages 129–132, 2005

2005
[12]

A statistical model for tensor pca

Andrea Montanari and Emile Richard. A statistical model for tensor pca. InAdvances in Neural Information Processing Systems, volume 27, 2014

2014
[13]

Eigenvalues of a real supersymmetric tensor.Journal of Symbolic Computation, 40 (6):1302–1324, 2005

Liqun Qi. Eigenvalues of a real supersymmetric tensor.Journal of Symbolic Computation, 40 (6):1302–1324, 2005

2005
[14]

Spectral norm of random tensors, 2014

Ryota Tomioka and Taiji Suzuki. Spectral norm of random tensors, 2014. arXiv:1407.1870

2014 arXiv
[15]

Sharp analysis of power iteration for tensor pca.Journal of Machine Learning Research, 25(195):1–42, 2024

Yuchen Wu and Kangjie Zhou. Sharp analysis of power iteration for tensor pca.Journal of Machine Learning Research, 25(195):1–42, 2024

2024
[16]

Yin, Zhidong D

Yong Q. Yin, Zhidong D. Bai, and Pathak R. Krishnaiah. On the limit of the largest eigenvalue of the large dimensional sample covariance matrix.Probability Theory and Related Fields, 78(4): 509–521, 1988. 67

1988

Pith tools

Reviewed June 28, 2026 · model on record in the stance chip above.

[1] [1]

Kakade, and Matus Telgarsky

Animashree Anandkumar, Rong Ge, Daniel Hsu, Sham M. Kakade, and Matus Telgarsky. Tensor decompositions for learning latent variable models.Journal of Machine Learning Research, 15 (80):2773–2832, 2014

2014

[2] [2]

Random matrices and complexity of spin glasses.Communications on Pure and Applied Mathematics, 66(2):165–201, 2013

Antonio Auffinger, Gérard Ben Arous, and Jiří Černý. Random matrices and complexity of spin glasses.Communications on Pure and Applied Mathematics, 66(2):165–201, 2013

2013

[3] [3]

Bai, Jack W

Zhidong D. Bai, Jack W. Silverstein, and Yong Q. Yin. A note on the largest eigenvalue of a large dimensional sample covariance matrix.Journal of Multivariate Analysis, 26(2):166–168, 1988

1988

[4] [4]

The landscape of the spiked tensor model.Communications on Pure and Applied Mathematics, 72(11):2282–2330, 2019

Gérard Ben Arous, Song Mei, Andrea Montanari, and Mihai Nica. The landscape of the spiked tensor model.Communications on Pure and Applied Mathematics, 72(11):2282–2330, 2019

2019

[5] [5]

A multilinear singular value decomposition.SIAM Journal on Matrix Analysis and Applications, 21(4):1253–1278, 2000

Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. A multilinear singular value decomposition.SIAM Journal on Matrix Analysis and Applications, 21(4):1253–1278, 2000

2000

[6] [6]

Hopkins, Jonathan Shi, and David Steurer

Samuel B. Hopkins, Jonathan Shi, and David Steurer. Tensor principal component analysis via sum-of-squares proofs. InProceedings of the 28th Conference on Learning Theory, volume 40 of Proceedings of Machine Learning Research, pages 956–1006, 2015

2015

[7] [7]

Hopkins, Tselil Schramm, Jonathan Shi, and David Steurer

Samuel B. Hopkins, Tselil Schramm, Jonathan Shi, and David Steurer. Fast spectral algorithms from sum-of-squares proofs: tensor decomposition and planted sparse vectors. InProceedings of the Forty-Eighth Annual ACM Symposium on Theory of Computing, pages 178–191, 2016. 66

2016

[8] [8]

Huang, Qing Yang, and Guang Cheng

Jiaoyang Huang, Daniel Z. Huang, Qing Yang, and Guang Cheng. Power iteration for tensor pca.Journal of Machine Learning Research, 23(128):1–47, 2022

2022