REVIEW 2 major objections 4 minor 43 references
Randomized algorithms for streaming low-rank approximation in tree tensor network format
T0 review · 2 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read Randomized Nyström sketching gives single-pass low-rank tree tensor network approximation with error controlled by the best lower-rank fit.
desk verdict TTNN is a solid new streaming method for tree tensor networks; the STTNN error bound is false as stated. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the per-node oblique Nyström projector $P_{\ell,k} = T_{\ell,k}X_{\ell,k}(Y_{\ell,k}^T T_{\ell,k}X_{\ell,k})^+ Y_{\ell,k}^T$, built from one unfolding $T_{\ell,k}$ of the tensor and two random dimension-reduction maps per node of the index tree. The approximant is the Kronecker product of these projectors acting on the vectorized tensor, level by level from leaves to root. This object carries the argument because the error telescopes into a sum of terms $(I-P_{\ell,k})T_{\ell,k}$, each bounded by the generalized Nyström matrix bounds; the recovery phase never forms the pseudoinverse explicitly, instead QR-factorizing the small sketch $Y_{\ell,k}^T T_{\ell,k} X_{\ell,k}$ and solving least-squares problems. For STTNN, the same projectors are built from pre-contracted tensors $T^S_{\ell,k}$, which is what lowers the cost on dense inputs.
What would settle it
Compute the STTNN approximant for a concrete tensor such as a 6D Hilbert tensor with mode size 20, evaluate the right-hand side of (6.11) exactly from the definitions of $\hat{V}$, $Q$, $X$, and $Y$ for many Gaussian draws, and compare it with the actual $\|T-\hat{T}\|_F$; a single draw violating the inequality would falsify Theorem 6.7 as stated.
Extended reading notes
Core claim
At the paper's core is the claim that the matrix generalized Nyström framework transfers to an arbitrary tree tensor network without losing its streaming property. For each node $I_{\ell,k}$ of the extended index tree, the algorithm defines an oblique projector $P_{\ell,k} = T_{\ell,k}X_{\ell,k}(Y_{\ell,k}^T T_{\ell,k}X_{\ell,k})^+ Y_{\ell,k}^T$ and applies all such projectors level by level, forming $\hat{T}^D = (\otimes_j P_{L,j})\cdots(\otimes_j P_{1,j})T^D$. Theorem 6.4 asserts that for any rank $\tilde{R}<R$, $\|T-\hat{T}\|_F \le \|T-T_{\tilde{R}}\|_F$ times a sum over nodes of factors $\rho_{\ell,k}\tau_{\ell,k}$ multiplied by products of $\eta$ factors, where $\rho$ and $\tau$ come from the two-step Nyström bounds for matrices and $\eta$ measures the squeezing of each projector; $T_{\tilde{R}}$ is any best TTN approximation of that lower rank. With independent Gaussian sketches, Theorem 6.6 upgrades this to an expected-error bound whose constants depend only on ranks and oversampling. The paper also claims the sequential variant STTNN inherits the deterministic bound (Theorem 6.7), while noting that its sketches involve contracted tensors and that no expectation bound is provided. Numerical experiments support the qualitative claim that the randomized TTN fits track the accuracy of a deterministic TTN-SVD while using far less time, with STTNN fastest on dense tensors.
Load-bearing premise
For STTNN, the deterministic error guarantee is asserted without a proof: the TTNN analysis is said to adapt with minimal modifications, but the quantities $\rho$, $\tau$, and $\eta$ are defined from full unfoldings $T_{\ell,k}$, while the STTNN projectors are built from contracted tensors $T^S_{\ell,k}$; if that transfer fails, the STTNN error bound is not established.
Editorial extensions
If this is right
- Any tensor that can be visited once can receive a rank-$R$ tree tensor network approximation without revisiting entries, with error controlled by the best lower-rank TTN approximation.
- Because all node sketches in TTNN are independent, the sketch phase is parallelizable; for a tensor expressed as a linear combination of streamed tensors, only the small sketch matrices need to be accumulated.
- The same algorithm doubles as a rounding routine for tensors already in TTN format: with structured Khatri-Rao sketches, the cost of computing transfer tensors scales linearly in mode sizes, replacing the SVD truncation of TTN-SVD.
- Choosing the two extreme index trees recovers the multilinear Nyström method for Tucker format and the streaming tensor train approximation, so the analysis unifies those existing formats.
- STTNN offers a cheaper, still single-pass route for dense tensors, at the price of losing the expected-error guarantee and cross-node parallelizability that TTNN enjoys.
Reading between the lines
- Editorial inference: if the deterministic STTNN bound in Theorem 6.7 is supplied with a full proof, the obstacle to an expectation bound is the correlation among sequential sketches rather than a fundamental gap in the projection idea; conditioning on earlier contractions could plausibly yield Gaussian STTNN estimates.
- Editorial inference: the same per-node two-sided projection recipe is format-agnostic, so extending it to cyclic tensor networks such as tensor ring or MERA would require choosing a tree cover or sweep order and defining ranks on cut edges; the paper identifies such extensions as open.
- Editorial inference: because the sketch phase is linear and local to each node, a testable distributed implementation could have each streamed term update only local sketches and communicate small matrices between nodes, enabling federated tensor approximation without materializing the full array.
- Editorial inference: the error bound's dependence on the number of nodes through products of $\eta$ factors suggests that balanced trees, which keep the tree shallow, should be preferred in practice; the paper's experiments use one fixed shallow tree and leave this dependence untested.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes two randomized algorithms for streaming low-rank approximation of tensors in a general tree tensor network format. The tree tensor network Nystrom (TTNN) method applies generalized Nystrom oblique projectors built from two-sided sketches at every non-root node of the index tree; the sequential variant STTNN reuses contractions with previously drawn sketches to reduce cost. The paper proves a deterministic error bound for TTNN (Theorem 6.4), a Gaussian expectation bound (Theorem 6.6), and states a deterministic bound for STTNN (Theorem 6.7). The final sections discuss structured Khatri-Rao sketchings for TTN rounding and present numerical experiments on dense and structured tensors.
Significance. If the TTNN analysis is correct, the paper gives a genuinely unifying framework: it contains the multilinear Nystrom and streaming tensor-train approximation as special cases, retains single-pass/streamable features, and provides explicit deterministic and Gaussian bounds. The release of code for the experiments is a concrete strength. The STTNN result, however, is a major unresolved problem: Theorem 6.7 is not a minor adaptation of the TTNN proof, and as stated it is ill-posed. Since STTNN is one of the two headline algorithms, the current theoretical contribution is only partial.
major comments (2)
- [6.3, Theorem 6.7 and Eq. (5.4)] The claim in Section 6.3 that the TTNN deterministic analysis can be adapted to STTNN with minimal modifications is not supported. In STTNN the projector in (5.4) is built from A = T^S_(l,k), the tensor already contracted along previously processed indices, while the error term that must be controlled involves the original T_(l,k). The generalized Nystrom bounds (2.1)-(2.2) apply when the sketch and the target coincide; they do not control (I-P)B when P is constructed from a matrix different from B. The theorem is also not well-formed: after a contraction, X_(l,k) has fewer rows than n_(D\I_(l,k)), so the product Vbar_(l,k)^T X_(l,k) in the definition of rho_(l,k) is dimensionally incompatible with Vbar_(l,k) being a right singular vector matrix of the original T_(l,k). Concretely, for T = diag(10,1,0.1) with two leaves, R = (2,2), pR = (1,1), and the full-column-rank choices X1 = Y1 = [e2,e3], X2 = I2, Y2 = [e2,e3], the STTNN approximant is diag(0,1,0.1), whose Frobenius error is 10, while ||T - T_pR||_F is approximately 1.005. The theorem cannot even be applied to this example because Vbar_(1,2)^T X_(1,2) is undefined; the dimensional mismatch is therefore not a cosmetic issue. Theorem 6.7 should be removed or replaced by a statement with a proof and with definitions compatible with the contracted sketch dimensions.
- [6.1, Theorem 6.4 and Lemma 6.1] The statement of Theorem 6.4 does not include the full-column-rank hypothesis under which Lemma 6.1 is proved. The proof of the theorem uses the identity P = Q(Y^T Q)^+ Y^T and the inequality (6.1), both of which require Y^T_(l,k) T_(l,k) X_(l,k) to have full column rank. Without this hypothesis the deterministic bound need not hold for arbitrary user-supplied DRMs. Please state the rank condition explicitly in Theorems 6.4 and 6.7, or treat the rank-deficient case separately.
minor comments (4)
- [2, Eqs. (2.1)-(2.2)] The displayed bounds for HMT and GN omit the square-root factors that are present in the correct statements and that are used later in the definitions of rho_(l,k) and tau_(l,k); please update the display.
- [8, second experiment] The paragraph beginning with In the second experiment is repeated verbatim; remove the duplicate.
- [5, Eqs. (5.1)-(5.4)] The contraction convention for T times Y^T_(l,k) is introduced informally; a formal definition of the contracted tensor T^S_(l,k) and of the row count of X_(l,k) would prevent dimension errors such as the one affecting Theorem 6.7.
- [3, Definition 3.4] The use of the same symbol I for the index tree and its extension is confusing; please use a different notation for the extended tree.
Circularity Check
No significant circularity: TTNN bounds are derived from external GN and Gaussian estimates, and the STTNN gap is an omitted proof, not a circular reduction.
full rationale
The central derivation chain is self-contained with respect to independent inputs. Lemma 6.1 reduces the oblique projector to Q(Y^T Q)^+ Y^T and bounds ||P B||_F by ||B||_F times a Y-sketch factor; this is an algebraic consequence of the projector definition (4.5), not an input. Lemma 6.3 telescopes the chain of projectors from Lemma 6.2, and Theorem 6.4 invokes the generalized Nyström bounds (2.1)-(2.2) from the external reference [24]; the factors rho, tau, and eta are norms of sketching matrices, not fitted parameters. Theorem 6.6 iterates Lemma 6.5, whose Gaussian expectation estimate is taken from external propositions in [12], and compares the algorithm output to the best rank-pR TTN error through standard tail bounds; no data are used to calibrate the constants. The TTNN approximant is defined by the projectors (4.5), so the error bound is a statement about that definition rather than an equivalent restatement of it. The only questionable passage is Section 6.3 and Theorem 6.7, where the deterministic STTNN bound is asserted to follow "with minimal modifications" and the proof is omitted; that is a missing proof or correctness risk, not a circular reduction of the claimed bound to its own assumptions. The self-citations [28], [29], and [42] appear as related-work and application context and are not load-bearing for any theorem. Therefore no circular step is established.
Assumptions & free parameters
assumptions (5)
- standard math Gaussian GN expected error bound (2.4), E_GN ≤ E_SVD sqrt(1+pr/(r-pr-1)) sqrt(1+r/(p-1))
- standard math The HMT/GN deterministic error bounds in the forms used for ρ and τ (sqrt(1+||...||^2) factors)
- domain assumption Full column rank of Y^T_{ℓ,k} T_{ℓ,k} X_{ℓ,k} for every node
- standard math Best-TTN-approximation error dominates each unfolding tail error (Hackbusch [37, Theorem 11.6])
- standard math Rotational invariance and moment bounds for standard Gaussian matrices (Halko et al. [12, Prop 10.1-10.2])
Cite this review
Pith. "Pith review of Randomized algorithms for streaming low-rank approximation in tree tensor network format." pith.science (2026). https://pith.science/paper/V32KLOMW
@misc{pith2026241206111,
author = {Pith},
title = {Pith review of: Randomized algorithms for streaming low-rank approximation in tree tensor network format},
year = {2026},
howpublished = {\url{https://pith.science/paper/V32KLOMW}},
note = {Machine review of arXiv:2412.06111}
}
read the original abstract
In this work, we present the tree tensor network Nystr\"om (TTNN), an algorithm that extends recent research on streamable tensor approximation, such as for Tucker and tensor-train formats, to the more general tree tensor network format, enabling a unified treatment of various existing methods. Our method retains the key features of the generalized Nystr\"om approximation for matrices, that is randomized, single-pass, streamable, and cost-effective. Additionally, the structure of the sketching allows for parallel implementation. We provide a deterministic error bound for the algorithm and, in the specific case of Gaussian dimension reduction maps, also a probabilistic one. We also introduce a sequential variant of the algorithm, referred to as sequential tree tensor network Nystr\"om (STTNN), which offers better performance for dense tensors. Furthermore, both algorithms are well-suited for the recompression or rounding of tensors in the tree tensor network format. Numerical experiments highlight the efficiency and effectiveness of the proposed methods.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[24]
Fast and stable randomized low-rank matrix approximation
Yuji Nakatsukasa. Fast and stable randomized low-rank matrix approximation. arXiv preprint arXiv:2009.11392, 2020
arXiv 2009
-
[1]
Tamara G. Kolda and Brett W. Bader. Tensor decomposition s and applications. SIAM Rev. , 51(3):455–500, 2009
work page 2009
- [2]
-
[3]
Hierarchical singular value decomposi tion of tensors
Lars Grasedyck. Hierarchical singular value decomposi tion of tensors. SIAM J. Matrix Anal. Appl., 31(4):2029–2054, 2009/10
work page 2009
-
[4]
Classical sim ulation of quantum many-body systems with a tree tensor network
Yaoyun Shi, Luming Duan, and Guifre Vidal. Classical sim ulation of quantum many-body systems with a tree tensor network. Physical Review A—Atomic, Molecular, and Optical Physics, 74(2):022320, 2006
work page 2006
-
[5]
Simulating strongly correlated quantum systems with tree tensor networks
Valentin Murg, Frank Verstraete, ¨Ors Legeza, and Reinhard M Noack. Simulating strongly correlated quantum systems with tree tensor networks. Physical Review B—Condensed Matter and Materials Physics , 82(20):205105, 2010
work page 2010
-
[6]
Valentin Murg, Frank Verstraete, Reinhold Schneider, P eter R Nagy, and O Legeza. Tree tensor network state with variable tensor order: An efficient multir eference method for strongly correlated systems. Journal of Chemical Theory and Computation , 11(3):1027–1036, 2015
work page 2015
-
[7]
Ti me integration of tree tensor net- works
Gianluca Ceruti, Christian Lubich, and Hanna W alach. Ti me integration of tree tensor net- works. SIAM Journal on Numerical Analysis , 59(1):289–313, 2021
work page 2021
Show all 43 references
-
[8]
Ra nk-adaptive time integration of tree tensor networks
Gianluca Ceruti, Christian Lubich, and Dominik Sulz. Ra nk-adaptive time integration of tree tensor networks. SIAM Journal on Numerical Analysis , 61(1):194–222, 2023
2023
-
[9]
Griffiths singularities in the ran- dom quantum Ising antiferromagnet: A tree tensor network re normalization group study
Yu-Ping Lin, Ying-Jer Kao, Pochung Chen, and Yu-Cheng Li n. Griffiths singularities in the ran- dom quantum Ising antiferromagnet: A tree tensor network re normalization group study. Physical Review B , 96(6):064427, 2017
2017
-
[10]
Tree tensor network approach to sim ulating Shor’s algorithm
Eugene Dumitrescu. Tree tensor network approach to sim ulating Shor’s algorithm. Physical Review A , 96(6):062322, 2017
2017
-
[11]
Tree tens or networks for generative modeling
Song Cheng, Lei W ang, Tao Xiang, and Pan Zhang. Tree tens or networks for generative modeling. Physical Review B , 99(15):155131, 2019
2019
-
[12]
Nathan Halko, Per-Gunnar Martinsson, and Joel A. Tropp . Finding structure with randomness: probabilistic algorithms for constructing approximate ma trix decompositions. SIAM Rev. , 53(2):217–288, 2011
2011
-
[13]
A multilinear singular value decomposition
Lieven De Lathauwer, Bart De Moor, and Joos Vandewalle. A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. , 21(4):1253–1278, 2000
2000
-
[14]
A new truncation strategy for the higher-order singular value decomposition
Nick Vannieuwenhoven, Raf Vandebril, and Karl Meerber gen. A new truncation strategy for the higher-order singular value decomposition. SIAM J. Sci. Comput. , 34(2):A1027–A1052, 2012
2012
-
[15]
Saibaba, and Misha E
Rachel Minster, Arvind K. Saibaba, and Misha E. Kilmer. Randomized algorithms for low-rank tensor decompositions in the Tucker format. SIAM J. Math. Data Sci. , 2(1):189–215, 2020
2020
-
[16]
Randomized algorithms for the approximations of Tucker and the tensor train decompositions
Maolin Che and Yimin W ei. Randomized algorithms for the approximations of Tucker and the tensor train decompositions. Adv. Comput. Math. , 45(1):395–428, 2019
2019
-
[17]
Reid, and Arvind K
Hussam Al Daas, Grey Ballard, Paul Cazeaux, Eric Hallma n, Agnieszka Miedlar, Mirjeta Pasha, Tim W. Reid, and Arvind K. Saibaba. Randomized algorithms fo r rounding in the tensor- train format. SIAM J. Sci. Comput. , 45(1):A74–A95, 2023
2023
-
[18]
RTSMS: Randomize d Tucker with single-mode sketch- ing
Behnam Hashemi and Yuji Nakatsukasa. RTSMS: Randomize d Tucker with single-mode sketch- ing. arXiv preprint arXiv:2311.14873 , 2023
2023 arXiv
-
[19]
J. A. Tropp. User-friendly tail bounds for sums of rando m matrices. Found. Comput. Math. , 2011
2011
-
[20]
High-dimensional probability
Roman Vershynin. High-dimensional probability. an in troduction with applications in data science. Cambridge University Press , 2018
2018
-
[21]
Joel A. Tropp. Improved analysis of the subsampled rand omized Hadamard transform. Adv. Adapt. Data Anal. , 3(1-2):115–126, 2011
2011
-
[22]
New and improved Johnson –Lindenstrauss embeddings via the restricted isometry property
Felix Krahmer and Rachel W ard. New and improved Johnson –Lindenstrauss embeddings via the restricted isometry property. SIAM Journal on Mathematical Analysis , 43(3):1269– 1281, 2011
2011
-
[23]
Near-optimal hierarchical matrix approxim ation from matrix-vector prod- ucts
Tyler Chen, Feyza Duman Keles, Diana Halikias, Cameron Musco, Christopher Musco, and David Persson. Near-optimal hierarchical matrix approxim ation from matrix-vector prod- ucts. arXiv preprint arXiv:2407.04686 , 2024
2024 arXiv
-
[25]
Clarkson and David P
Kenneth L. Clarkson and David P. W oodruff. Numerical lin ear algebra in the streaming model. In Proceedings of the forty-first annual ACM symposium on Theor y of computing , pages 205–214, 2009
2009
-
[26]
Tropp, Alp Yurtsever, Madeleine Udell, and Volk an Cevher
Joel A. Tropp, Alp Yurtsever, Madeleine Udell, and Volk an Cevher. Practical sketching algo- rithms for low-rank matrix approximation. SIAM Journal on Matrix Analysis and Appli- cations, 38(4):1454–1485, 2017
2017
-
[27]
Low-rank Tucker approximation of a tensor from streaming data
Yiming Sun, Yang Guo, Charlene Luo, Joel Tropp, and Made leine Udell. Low-rank Tucker approximation of a tensor from streaming data. SIAM J. Math. Data Sci. , 2(4):1123– 1150, 2020. This manuscript is for review purposes only. 26 A. BUCCI, G. VERZELLA
2020
-
[28]
A multilinear Nystr¨ om algorithm for low-rank approxi- mation of tensors in Tucker format
Alberto Bucci and Leonardo Robol. A multilinear Nystr¨ om algorithm for low-rank approxi- mation of tensors in Tucker format. SIAM Journal on Matrix Analysis and Applications , 45(4):1929–1953, 2024
1929
-
[29]
A sequential multili near Nystr¨ om algorithm for stream- ing low-rank approximation of tensors in Tucker format
Alberto Bucci and Behnam Hashemi. A sequential multili near Nystr¨ om algorithm for stream- ing low-rank approximation of tensors in Tucker format. Applied Mathematics Letters , 159:109271, 2024
2024
-
[30]
Streaming tensor train approximation
Daniel Kressner, Bart Vandereycken, and Rik Voorhaar. Streaming tensor train approximation. SIAM Journal on Scientific Computing , 45(5):A2610–A2631, 2023
2023
-
[31]
Clarkson and David P
Kenneth L. Clarkson and David P. W oodruff. Low-rank appr oximation and regression in input sparsity time. Journal of the ACM (JACM) , 63(6):1–45, 2017
2017
-
[32]
Improved matrix a lgorithms via the subsampled random- ized Hadamard transform
Christos Boutsidis and Alex Gittens. Improved matrix a lgorithms via the subsampled random- ized Hadamard transform. SIAM J. Matrix Anal. Appl. , 34(3):1301–1340, 2013
2013
-
[33]
A fast randomized alg orithm for overdetermined linear least-squares regression
Vladimir Rokhlin and Mark Tygert. A fast randomized alg orithm for overdetermined linear least-squares regression. Proc. Natl. Acad. Sci. USA , 105(36):13212–13217, 2008
2008
-
[34]
Per-Gunnar Martinsson and Joel A. Tropp. Randomized nu merical linear algebra: foundations and algorithms. Acta Numer. , 29:403–572, 2020
2020
-
[35]
Tracking tensor ring decompositi ons of streaming tensors, 2023
Yajie Yu and Hanyu Li. Tracking tensor ring decompositi ons of streaming tensors, 2023
2023
-
[36]
On the optimal linear contrac- tion order of tree tensor networks, and beyond
Mihail Stoian, Richard M Milbradt, and Christian B Mend l. On the optimal linear contrac- tion order of tree tensor networks, and beyond. SIAM Journal on Scientific Computing , 46(5):B647–B668, 2024
2024
-
[37]
Tensor Spaces and Numerical Tensor Calculus , volume 56 of Springer Series in Computational Mathematics
W olfgang Hackbusch. Tensor Spaces and Numerical Tensor Calculus , volume 56 of Springer Series in Computational Mathematics . Springer Berlin Heidelberg, 2 edition, 2019
2019
-
[38]
Cost-efficient Gaussian tensor network embeddings for tensor-structured inputs
Linjian Ma and Edgar Solomonik. Cost-efficient Gaussian tensor network embeddings for tensor-structured inputs. Advances in Neural Information Processing Systems , 35:38980– 38993, 2022
2022
-
[39]
Ahle, Michael Kapralov, Jakob B.T
Thomas D. Ahle, Michael Kapralov, Jakob B.T. Knudsen, R asmus Pagh, Ameya Velingker, David P. W oodruff, and Amir Zandieh. Oblivious sketching of h igh-degree polynomial kernels. In Proceedings of the Fourteenth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 141–160. S...
2020
-
[40]
Mahankali, David P
Arvind V. Mahankali, David P. W oodruff, and Ziyu Zhang. N ear-linear time and fixed-parameter tractable algorithms for tensor decomposi tions. arXiv preprint arXiv:2207.07417, 2022
2022 arXiv
-
[41]
A randomized tensor train singular value decomposition
Benjamin Huber, Reinhold Schneider, and Sebastian W ol f. A randomized tensor train singular value decomposition. In Compressed Sensing and its Applications: Second Internati onal MATHEON Conference 2015 , pages 261–290. Springer, 2017
2015
-
[42]
Ran domized sketched TT-GMRES for linear systems with tensor structure
Alberto Bucci, Davide Palitta, and Leonardo Robol. Ran domized sketched TT-GMRES for linear systems with tensor structure. arXiv preprint arXiv:2409.09471 , 2024
2024 arXiv
-
[43]
TT-GMRES: solution to a linear system in the structured tensor format
Sergey Dolgov. TT-GMRES: solution to a linear system in the structured tensor format. Russian Journal of Numerical Analysis and Mathematical Mod elling, 28(2):149–172, 2013. This manuscript is for review purposes only
2013
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.