REVIEW 2 major objections 4 minor 28 references
Faster Tensor Train Decomposition for Sparse Data
T0 review · 2 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A sparse tensor can be TT-decomposed in time set by its nonzero fibers, not its full size.
desk verdict FastTT is a genuinely useful sparse TT algorithm with proven error bounds and big speedups, but its advertised 'ranks not larger than TT-SVD' claim is false as stated and needs fixing. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
A $p$-fiber is a one-dimensional slice of a tensor obtained by fixing all indices except the $p$-th. The load-bearing construction is Theorem 3: a sparse tensor with $R$ nonzero $p$-fibers is exactly a sum of $R$ rank-1 tensors, hence an exact TT of rank $R$ whose cores are sparse and have quasi-permutation unfoldings (each column contains a single 1). Parallel-vector rounding eliminates duplicate columns of these quasi-permutation matrices in roughly linear time and without floating-point arithmetic, and the revised TT-rounding replaces the standard right-to-left/left-to-right sweeps with two sweeps starting from the fiber direction $p$, using the orthonormality of the outer cores to keep the error budget clean.
What would settle it
Construct a sparse tensor with a known exact TT decomposition of small rank, run FastTT with a tolerance near machine precision, and check whether the returned TT ranks equal those from TT-SVD and the relative error stays below the tolerance; any mismatch would refute the central correctness and rank claims.
Extended reading notes
Core claim
On its own terms, the paper establishes Theorem 9: for a sparse $d$-way tensor $A$ and any tolerance $\varepsilon$, the FastTT algorithm returns a TT-format tensor $B$ with $\|A-B\|_F \le \varepsilon \|A\|_F$, and when $\varepsilon$ is near machine precision the TT ranks of $B$ are exactly those produced by TT-SVD. The exact rank match follows because the initial TT built from nonzero $p$-fibers has rank $R$, and the later rounding steps reduce ranks without exceeding the allocated error budget. The complexity claim is expressed in the FLOP estimate (13): the dominant cost is a small number of SVDs whose smaller dimension is bounded by $R$ and by the prefix or suffix dimension products, so sparsity, not tensor volume, controls the running time. This is what lets the algorithm decompose tensors that TT-SVD cannot handle and convert large sparse matrices into matrix product operators.
Load-bearing premise
The runtime advantage, though not the correctness guarantee, rests on the number $R$ of nonzero fibers being small compared with the full tensor size; if $R$ approaches the total number of entries, the initial TT rank grows and the speedup over TT-SVD mostly disappears.
Editorial extensions
If this is right
- Tensors with many dimensions but few nonzero entries can be decomposed without ever forming a dense unfolding, so TT-based methods become practical for very high-order sparse data.
- Sparse matrices can be converted to matrix product operators with the same speed-up, allowing larger linear systems to be solved by alternating least squares or related TT-iterative methods.
- Setting the tolerance near zero makes the final TT ranks identical to TT-SVD's ranks, so existing pipelines can substitute FastTT without changing downstream behavior.
- The proven error bound, including the dynamic parameter variant, means the algorithm does not require per-mode truncation parameters to be hand-tuned.
Reading between the lines
- Because the nonzero-fiber representation is just a sum of rank-1 terms, the same construction could plausibly be carried over to tensor-ring or hierarchical Tucker formats; the paper develops it only for TT.
- A probabilistic extension that samples a subset of nonzero fibers first and then corrects by a residual sweep could trade a small accuracy loss for still lower initial ranks on extremely sparse, high-dimensional tensors; the paper does not explore this.
- The quasi-permutation observation may apply beyond sparse fibers: any TT core whose unfoldings are quasi-permutation matrices could be rounded by the same duplicate-column elimination, which might speed up other structured tensor computations.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FastTT, an algorithm for computing a tensor-train (TT) decomposition of a large sparse tensor. It first constructs an exact TT whose initial rank is the number R of nonzero p-fibers (Algorithm 4), then applies a parallel-vector rounding procedure and a two-sided TT-rounding procedure (Algorithm 6) to reduce the rank while satisfying a prescribed relative error bound. A dynamic truncation variant (Algorithm 8) is also proposed. The authors prove an error bound (Theorem 9), give FLOP-based complexity estimates, and report speedups on image/video inpainting, finite-difference matrices, and a road-network adjacency matrix.
Significance. The core algorithmic idea is attractive: by rearranging a sparse tensor as a sum of rank-one tensors and exploiting the resulting quasi-permutation structure, FastTT avoids forming dense unfolded matrices and can be much faster than TT-SVD on data with few nonzero fibers. The code is publicly available, and the main error-bound proof is mostly self-contained. However, the paper's advertised rank guarantee relative to TT-SVD is contradicted by its own experiments, and one auxiliary proof is incomplete. These issues do not invalidate the speed advantage, but they materially affect what the paper can honestly claim.
major comments (2)
- [Section 3.3 (paragraph after Algorithm 7) and Table 3] The claim that FastTT yields TT ranks no larger than those of TT-SVD for the same tolerance is contradicted by Table 3. For N=10^4 and ε=1e-2, TT-SVD returns ranks (28,1390,70) while FastTT returns (28,1395,70). The supporting paragraph only proves rank equality for ε=0 and p=1, then asserts 'similar results' without a proof. For 1<p<d, the thresholds in Eq. (11) are δ_k = ε||A||_F/(√(p-1)+√(d-p)), which are strictly smaller than TT-SVD's ε||A||_F/√(d-1), so Algorithm 6 may retain more singular values than TT-SVD. This invalidates the Introduction's claim of producing the same compact representation 'without ... an increase of the TT ranks.' Please either prove a rank inequality under the actual thresholds or revise the claim and discuss the trade-off.
- [Section 3.2, Theorem 5] The proof of Theorem 5 is incomplete. It covers only k<p and dismisses k>p with 'the same line of reasoning can be used.' The two cases are not symmetric in an obvious way because the contraction order differs (G^{(k-1)} ×3 T^T versus G^{(k+1)} ×1 T) and the unfolding is transposed. Since Corollary 7 and hence Theorem 9 rely on Theorem 5, the correctness proof for general p is not fully rigorous as written. Please supply the complete argument for k>p.
minor comments (4)
- [Section 3.3, proof of Theorem 9] The displayed chain of equalities in the proof of Theorem 9 writes ||A-C||_F^2 = δ_p^2 + ... as an equality. Because the SVD truncation only guarantees ||E_p||_F ≤ δ_p, this should be an inequality (≤). The final error bound is unaffected, but the current presentation is misleading.
- [Abstract] The sentence 'it can be used to decomposes arbitrary high-dimensional tensor' contains a grammatical error ('decomposes' should be 'decompose').
- [Section 3.3 and Section 4] The complexity estimates in Eqs. (1) and (13) rely on an unspecified constant C_SVD in f_SVD(m,n)=C_SVD mn min(m,n). It would be helpful to state explicitly that C_SVD is implementation-dependent and to explain how the FLOP comparisons should be interpreted across different SVD routines.
- [Table 3] The table would be easier to interpret if the difference between the FastTT and TT-SVD ranks were explicitly highlighted, especially for the ε=1e-2, N=10^4 case, since it directly contradicts the stated rank-guarantee claim.
Circularity Check
No circularity found: the FastTT construction and error bound are derived from standard TT algebra and external lemmas, with no fitted input called a prediction.
full rationale
The derivation chain is self-contained and externally anchored. Theorem 3 is a direct construction: the sparse tensor is written as the sum of R rank-1 tensors, and Lemma 2 (from Oseledets' TT paper, [3]) gives the TT cores of the sum; the proof does not assume the target result. Algorithm 4's correctness follows from Theorem 3 and associativity of contractions, and its rank bounds are explicit upper bounds in Eq. (10), not experimental fits. Algorithm 6's error bound (Theorem 9) is proved from Corollary 7 and Lemma 8; Corollary 7 is proved in the paper from the quasi-permutation structure, and Lemma 8 is an independent external lemma cited to [14, Appendix B]. The truncation parameters in Eq. (11) are chosen so that the summed square errors satisfy the prescribed epsilon bound, not fitted to the reported outputs. Algorithm 8's correctness is proved by a loop invariant in Appendix A. Self-citations to the authors' prior work (e.g. [8], [11], [13], [16]) appear only in application/contextual sentences and are not load-bearing for correctness or complexity. The Table 3 discrepancy between FastTT and TT-SVD ranks at epsilon=1e-2 (28,1390,70 vs 28,1395,70) concerns the truth of the paper's rank-equality claim, not circularity: that claim is not used as an input to the derivation. No step reduces, by definition or by self-citation chain, to its own inputs.
Assumptions & free parameters
assumptions (6)
- standard math For a matrix M and an accuracy tolerance delta, the truncated SVD yields U, Sigma, V with ||M - U Sigma V^T||_F <= delta.
- standard math The Frobenius norm is invariant under multiplication by an orthonormal matrix.
- standard math Lemma 2 of Oseledets [3]: the TT cores of a sum of two TT tensors are block-diagonal concatenations of the cores.
- standard math Lemma 8 of Wang et al. [14, Appendix B]: if unfold2(U(i)) is orthonormal for i=1..k, then the unfolding of the first j cores is orthonormal.
- domain assumption FLOP count of economic SVD on an m by n matrix is C_SVD mn min(m,n) with a universal constant C_SVD.
- domain assumption The sparse tensor A is accessible in a format that enumerates nonzero p-fibers efficiently via hash tables or balanced binary search trees.
Cite this review
Pith. "Pith review of Faster Tensor Train Decomposition for Sparse Data." pith.science (2026). https://pith.science/paper/AZZPNEJS
@misc{pith2026190802721,
author = {Pith},
title = {Pith review of: Faster Tensor Train Decomposition for Sparse Data},
year = {2026},
howpublished = {\url{https://pith.science/paper/AZZPNEJS}},
note = {Machine review of arXiv:1908.02721}
}
read the original abstract
In recent years, the application of tensors has become more widespread in fields that involve data analytics and numerical computation. Due to the explosive growth of data, low-rank tensor decompositions have become a powerful tool to harness the notorious curse of dimensionality. The main forms of tensor decomposition include CP decomposition, Tucker decomposition, tensor train (TT) decomposition, etc. Each of the existing TT decomposition algorithms, including the TT-SVD and randomized TT-SVD, is successful in the field, but neither can both accurately and efficiently decompose large-scale sparse tensors. Based on previous research, this paper proposes a new quasi-best fast TT decomposition algorithm for large-scale sparse tensors with proven correctness and the upper bound of its complexity is derived. In numerical experiments, we verify that the proposed algorithm can decompose sparse tensors faster than the TT-SVD, and have more speed, precision and versatility than randomized TT-SVD, and it can be used to decomposes arbitrary high-dimensional tensor without losing efficiency when the number of non-zero elements is limited. The new algorithm implements a large-scale sparse matrix TT decomposition that was previously unachievable, enabling tensor decomposition based algorithms to be applied in larger-scale scenarios.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
The expression of a tensor or a polyadic as a sum of products
Frank L Hitchcock. The expression of a tensor or a polyadic as a sum of products. Studies in Applied Mathematics , 6(1-4):164–189, 1927
work page 1927
-
[2]
Some mathematical notes on three-mode factor analysis
Ledyard R Tucker. Some mathematical notes on three-mode factor analysis. Psychometrika, 31(3):279–311, 1966
1966
-
[3]
Ivan V Oseledets. Tensor-train decomposition. SIAM Journal on Scientific Computing, 33(5):2295–2317, 2011
work page 2011
-
[4]
Solution of linear systems and ma- trix inversion in the TT-format
Ivan V Oseledets and SV Dolgov. Solution of linear systems and ma- trix inversion in the TT-format. SIAM Journal on Scientific Computing , 34(5):A2718–A2739, 2012. 26
work page 2012
-
[5]
Zheng Zhang, Xiu Yang, Ivan V Oseledets, George E Karniadakis, and Luca Daniel. Enabling high-dimensional hierarchical uncertainty quantifi- cation by ANOVA and tensor-train decomposition. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 34(1):63–76, 2014
work page 2014
-
[6]
Model reduction and simu- lation of nonlinear circuits via tensor decomposition
Haotian Liu, Luca Daniel, and Ngai Wong. Model reduction and simu- lation of nonlinear circuits via tensor decomposition. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 34(7):1059– 1069, 2015
work page 2015
-
[7]
Tensor computation: A new framework for high-dimensional problems in EDA
Zheng Zhang, Kim Batselier, Haotian Liu, Luca Daniel, and Ngai Wong. Tensor computation: A new framework for high-dimensional problems in EDA. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 36(4):521–536, 2017
work page 2017
-
[8]
Tensor network alter- nating linear scheme for MIMO Volterra system identification
Kim Batselier, Zhongming Chen, and Ngai Wong. Tensor network alter- nating linear scheme for MIMO Volterra system identification. Automatica, 84:26–35, 2017
work page 2017
Show all 28 references
-
[9]
Approximation of 2 d× 2d matrices using tensor decom- position
Ivan V Oseledets. Approximation of 2 d× 2d matrices using tensor decom- position. SIAM Journal on Matrix Analysis and Applications , 31(4):2130– 2145, 2010
2010
-
[10]
On low-rank approximability of solutions to high-dimensional operator equations and eigenvalue problems
Daniel Kressner and Andr´ e Uschmajew. On low-rank approximability of solutions to high-dimensional operator equations and eigenvalue problems. Linear Algebra and its Applications , 493:556–572, 2016
2016
-
[11]
Computing low-rank approximations of large-scale matrices with the tensor network randomized SVD
Kim Batselier, Wenjian Yu, Luca Daniel, and Ngai Wong. Computing low-rank approximations of large-scale matrices with the tensor network randomized SVD. SIAM Journal on Matrix Analysis and Applications , 39(3):1221–1244, 2018
2018
-
[12]
Efficient low rank tensor ring completion
Wenqi Wang, Vaneet Aggarwal, and Shuchin Aeron. Efficient low rank tensor ring completion. In IEEE International Conference on Computer Vision (ICCV), pages 5698–5706, 2017
2017
-
[13]
Fast and accurate tensor completion with total variation regularized tensor trains
Ching-Yun Ko, Kim Batselier, Wenjian Yu, and Ngai Wong. Fast and accurate tensor completion with total variation regularized tensor trains. arXiv preprint arXiv:1804.06128 , 2018
2018 arXiv
-
[14]
Tensor train neigh- borhood preserving embedding
Wenqi Wang, Vaneet Aggarwal, and Shuchin Aeron. Tensor train neigh- borhood preserving embedding. IEEE Transactions on Signal Processing , 66(10):2724–2732, 2018
2018
-
[15]
Support vector machine based on low-rank tensor train decomposition for big data applications
Yongkang Wang, Weicheng Zhang, Zhuliang Yu, Zhenghui Gu, Hao Liu, Zhaoquan Cai, Congjun Wang, and Shihan Gao. Support vector machine based on low-rank tensor train decomposition for big data applications. In 2017 12th IEEE Conference on Industrial Electronics and Applications ...
2017
-
[16]
Par- allelized tensor train learning of polynomial classifiers
Zhongming Chen, Kim Batselier, Johan AK Suykens, and Ngai Wong. Par- allelized tensor train learning of polynomial classifiers. IEEE transactions on neural networks and learning systems , 29(10):4621–4632, 2017
2017
-
[17]
Whole brain fMRI pattern analysis based on tensor neural network
Xiaowen Xu, Qiang Wu, Shuo Wang, Ju Liu, Jiande Sun, and Andrzej Ci- chocki. Whole brain fMRI pattern analysis based on tensor neural network. IEEE Access, 6:29297–29305, 2018
2018
-
[18]
TT-cross approximation for mul- tidimensional arrays
Ivan Oseledets and Eugene Tyrtyshnikov. TT-cross approximation for mul- tidimensional arrays. Linear Algebra and its Applications , 432(1):70–88, 2010
2010
-
[19]
Fast adaptive interpolation of multi-dimensional arrays in tensor train format
Dmitry Savostyanov and Ivan Oseledets. Fast adaptive interpolation of multi-dimensional arrays in tensor train format. In The 2011 International Workshop on Multidimensional (nD) Systems , pages 1–8. IEEE, 2011
2011
-
[20]
A randomized tensor train singular value decomposition
Benjamin Huber, Reinhold Schneider, and Sebastian Wolf. A randomized tensor train singular value decomposition. In Compressed Sensing and its Applications, pages 261–290. Springer, Cham, 2017
2017
-
[21]
Finding struc- ture with randomness: Probabilistic algorithms for constructing approxi- mate matrix decompositions
Nathan Halko, Per-Gunnar Martinsson, and Joel A Tropp. Finding struc- ture with randomness: Probabilistic algorithms for constructing approxi- mate matrix decompositions. SIAM review, 53(2):217–288, 2011
2011
-
[22]
A theory of pseudoskeleton approximations
Sergei A Goreinov, Eugene E Tyrtyshnikov, and Nickolai L Zamarashkin. A theory of pseudoskeleton approximations. Linear algebra and its appli- cations, 261(1-3):1–21, 1997
1997
-
[23]
Generic construction of efficient matrix product operators
C Hubig, IP McCulloch, and U Schollw¨ ock. Generic construction of efficient matrix product operators. Physical Review B, 95(3):035129, 2017
2017
-
[24]
Xerus - A general purpose tensor library, 2014–2017
Benjamin Huber and Sebastian Wolf. Xerus - A general purpose tensor library, 2014–2017
2014
-
[25]
Corrected one-site density matrix renormalization group and alternating minimal energy algorithm
Sergey V Dolgov and Dmitry V Savostyanov. Corrected one-site density matrix renormalization group and alternating minimal energy algorithm. In Numerical Mathematics and Advanced Applications-ENUMATH 2013 , pages 335–343. Springer, 2015
2013
-
[26]
SNAP Datasets: Stanford large network dataset collection, June 2015
Jure Leskovec and Andrej Krevl. SNAP Datasets: Stanford large network dataset collection, June 2015
2015
-
[27]
Alan George and Joseph W. Liu. Computer Solution of Large Sparse Pos- itive Definite. Prentice Hall Professional Technical Reference, 1981
1981
-
[28]
Introduction to Algorithms
Thomas H Cormen, Charles E Leiserson, Ronald L Rivest, and Clifford Stein. Introduction to Algorithms. MIT press, 2009. 28
2009
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.