Pith. sign in

REVIEW 2 major objections 4 minor 28 references

Faster Tensor Train Decomposition for Sparse Data

T0 review · 2 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read A sparse tensor can be TT-decomposed in time set by its nonzero fibers, not its full size.

desk verdict FastTT is a genuinely useful sparse TT algorithm with proven error bounds and big speedups, but its advertised 'ranks not larger than TT-SVD' claim is false as stated and needs fixing. read the letter →

arxiv 1908.02721 v2 pith:AZZPNEJS submitted 2019-08-07 math.NA cs.NA

classification math.NAcs.NA MSC 15A6965F55
keywords tensortraindecompositionsparseTT-roundingparallel-vectorroundingquasi-permutationmatrixproductoperatorlow-rankapproximation
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes an algorithm, FastTT, for computing a tensor-train (TT) decomposition of a large sparse tensor. The claim is that runtime is driven by the number $R$ of nonzero one-dimensional fibers rather than by the product of the tensor's dimensions, while the error bound and the final TT ranks match those of the standard TT-SVD algorithm. The construction writes the sparse tensor exactly as a TT of rank $R$ using its nonzero fibers, then lowers the ranks with a lossless quasi-permutation-aware rounding step followed by a revised TT-rounding that carries a proven error bound. Experiments report speedups over TT-SVD from about 2x to 240x depending on sparsity, and one matrix-conversion case where TT-SVD could not finish at all. The significance is that TT-based methods become usable for sparse high-dimensional tensors that dense or randomized SVD routes cannot handle.

What carries the argument

A $p$-fiber is a one-dimensional slice of a tensor obtained by fixing all indices except the $p$-th. The load-bearing construction is Theorem 3: a sparse tensor with $R$ nonzero $p$-fibers is exactly a sum of $R$ rank-1 tensors, hence an exact TT of rank $R$ whose cores are sparse and have quasi-permutation unfoldings (each column contains a single 1). Parallel-vector rounding eliminates duplicate columns of these quasi-permutation matrices in roughly linear time and without floating-point arithmetic, and the revised TT-rounding replaces the standard right-to-left/left-to-right sweeps with two sweeps starting from the fiber direction $p$, using the orthonormality of the outer cores to keep the error budget clean.

What would settle it

Construct a sparse tensor with a known exact TT decomposition of small rank, run FastTT with a tolerance near machine precision, and check whether the returned TT ranks equal those from TT-SVD and the relative error stays below the tolerance; any mismatch would refute the central correctness and rank claims.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes Theorem 9: for a sparse $d$-way tensor $A$ and any tolerance $\varepsilon$, the FastTT algorithm returns a TT-format tensor $B$ with $\|A-B\|_F \le \varepsilon \|A\|_F$, and when $\varepsilon$ is near machine precision the TT ranks of $B$ are exactly those produced by TT-SVD. The exact rank match follows because the initial TT built from nonzero $p$-fibers has rank $R$, and the later rounding steps reduce ranks without exceeding the allocated error budget. The complexity claim is expressed in the FLOP estimate (13): the dominant cost is a small number of SVDs whose smaller dimension is bounded by $R$ and by the prefix or suffix dimension products, so sparsity, not tensor volume, controls the running time. This is what lets the algorithm decompose tensors that TT-SVD cannot handle and convert large sparse matrices into matrix product operators.

Load-bearing premise

The runtime advantage, though not the correctness guarantee, rests on the number $R$ of nonzero fibers being small compared with the full tensor size; if $R$ approaches the total number of entries, the initial TT rank grows and the speedup over TT-SVD mostly disappears.

Editorial extensions

If this is right

  • Tensors with many dimensions but few nonzero entries can be decomposed without ever forming a dense unfolding, so TT-based methods become practical for very high-order sparse data.
  • Sparse matrices can be converted to matrix product operators with the same speed-up, allowing larger linear systems to be solved by alternating least squares or related TT-iterative methods.
  • Setting the tolerance near zero makes the final TT ranks identical to TT-SVD's ranks, so existing pipelines can substitute FastTT without changing downstream behavior.
  • The proven error bound, including the dynamic parameter variant, means the algorithm does not require per-mode truncation parameters to be hand-tuned.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the nonzero-fiber representation is just a sum of rank-1 terms, the same construction could plausibly be carried over to tensor-ring or hierarchical Tucker formats; the paper develops it only for TT.
  • A probabilistic extension that samples a subset of nonzero fibers first and then corrects by a residual sweep could trade a small accuracy loss for still lower initial ranks on extremely sparse, high-dimensional tensors; the paper does not explore this.
  • The quasi-permutation observation may apply beyond sparse fibers: any TT core whose unfoldings are quasi-permutation matrices could be rounded by the same duplicate-column elimination, which might speed up other structured tensor computations.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes FastTT, an algorithm for computing a tensor-train (TT) decomposition of a large sparse tensor. It first constructs an exact TT whose initial rank is the number R of nonzero p-fibers (Algorithm 4), then applies a parallel-vector rounding procedure and a two-sided TT-rounding procedure (Algorithm 6) to reduce the rank while satisfying a prescribed relative error bound. A dynamic truncation variant (Algorithm 8) is also proposed. The authors prove an error bound (Theorem 9), give FLOP-based complexity estimates, and report speedups on image/video inpainting, finite-difference matrices, and a road-network adjacency matrix.

Significance. The core algorithmic idea is attractive: by rearranging a sparse tensor as a sum of rank-one tensors and exploiting the resulting quasi-permutation structure, FastTT avoids forming dense unfolded matrices and can be much faster than TT-SVD on data with few nonzero fibers. The code is publicly available, and the main error-bound proof is mostly self-contained. However, the paper's advertised rank guarantee relative to TT-SVD is contradicted by its own experiments, and one auxiliary proof is incomplete. These issues do not invalidate the speed advantage, but they materially affect what the paper can honestly claim.

major comments (2)
  1. [Section 3.3 (paragraph after Algorithm 7) and Table 3] The claim that FastTT yields TT ranks no larger than those of TT-SVD for the same tolerance is contradicted by Table 3. For N=10^4 and ε=1e-2, TT-SVD returns ranks (28,1390,70) while FastTT returns (28,1395,70). The supporting paragraph only proves rank equality for ε=0 and p=1, then asserts 'similar results' without a proof. For 1<p<d, the thresholds in Eq. (11) are δ_k = ε||A||_F/(√(p-1)+√(d-p)), which are strictly smaller than TT-SVD's ε||A||_F/√(d-1), so Algorithm 6 may retain more singular values than TT-SVD. This invalidates the Introduction's claim of producing the same compact representation 'without ... an increase of the TT ranks.' Please either prove a rank inequality under the actual thresholds or revise the claim and discuss the trade-off.
  2. [Section 3.2, Theorem 5] The proof of Theorem 5 is incomplete. It covers only k<p and dismisses k>p with 'the same line of reasoning can be used.' The two cases are not symmetric in an obvious way because the contraction order differs (G^{(k-1)} ×3 T^T versus G^{(k+1)} ×1 T) and the unfolding is transposed. Since Corollary 7 and hence Theorem 9 rely on Theorem 5, the correctness proof for general p is not fully rigorous as written. Please supply the complete argument for k>p.
minor comments (4)
  1. [Section 3.3, proof of Theorem 9] The displayed chain of equalities in the proof of Theorem 9 writes ||A-C||_F^2 = δ_p^2 + ... as an equality. Because the SVD truncation only guarantees ||E_p||_F ≤ δ_p, this should be an inequality (≤). The final error bound is unaffected, but the current presentation is misleading.
  2. [Abstract] The sentence 'it can be used to decomposes arbitrary high-dimensional tensor' contains a grammatical error ('decomposes' should be 'decompose').
  3. [Section 3.3 and Section 4] The complexity estimates in Eqs. (1) and (13) rely on an unspecified constant C_SVD in f_SVD(m,n)=C_SVD mn min(m,n). It would be helpful to state explicitly that C_SVD is implementation-dependent and to explain how the FLOP comparisons should be interpreted across different SVD routines.
  4. [Table 3] The table would be easier to interpret if the difference between the FastTT and TT-SVD ranks were explicitly highlighted, especially for the ε=1e-2, N=10^4 case, since it directly contradicts the stated rank-guarantee claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the FastTT construction and error bound are derived from standard TT algebra and external lemmas, with no fitted input called a prediction.

full rationale

The derivation chain is self-contained and externally anchored. Theorem 3 is a direct construction: the sparse tensor is written as the sum of R rank-1 tensors, and Lemma 2 (from Oseledets' TT paper, [3]) gives the TT cores of the sum; the proof does not assume the target result. Algorithm 4's correctness follows from Theorem 3 and associativity of contractions, and its rank bounds are explicit upper bounds in Eq. (10), not experimental fits. Algorithm 6's error bound (Theorem 9) is proved from Corollary 7 and Lemma 8; Corollary 7 is proved in the paper from the quasi-permutation structure, and Lemma 8 is an independent external lemma cited to [14, Appendix B]. The truncation parameters in Eq. (11) are chosen so that the summed square errors satisfy the prescribed epsilon bound, not fitted to the reported outputs. Algorithm 8's correctness is proved by a loop invariant in Appendix A. Self-citations to the authors' prior work (e.g. [8], [11], [13], [16]) appear only in application/contextual sentences and are not load-bearing for correctness or complexity. The Table 3 discrepancy between FastTT and TT-SVD ranks at epsilon=1e-2 (28,1390,70 vs 28,1395,70) concerns the truth of the paper's rank-equality claim, not circularity: that claim is not used as an input to the derivation. No step reduces, by definition or by self-citation chain, to its own inputs.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

No free parameters are fit to data. The central correctness proof relies on standard SVD truncation properties and on two cited lemmas (Oseledets' Lemma 2, Wang et al.'s Lemma 8); those are treated as accepted background results. The FLOP model with an unspecified constant C_SVD is an assumption used for the p-selection heuristic and runtime estimation, not for correctness.

assumptions (6)
  • standard math For a matrix M and an accuracy tolerance delta, the truncated SVD yields U, Sigma, V with ||M - U Sigma V^T||_F <= delta.
    Used in Algorithm 1 and Algorithm 6; it is the basis of the error accumulation in Theorem 9.
  • standard math The Frobenius norm is invariant under multiplication by an orthonormal matrix.
    Used in the proof of Theorem 9 to strip off the orthogonal left cores and reduce to the right part.
  • standard math Lemma 2 of Oseledets [3]: the TT cores of a sum of two TT tensors are block-diagonal concatenations of the cores.
    Used in Section 3.1 to build the exact TT from the sum of rank-1 tensors.
  • standard math Lemma 8 of Wang et al. [14, Appendix B]: if unfold2(U(i)) is orthonormal for i=1..k, then the unfolding of the first j cores is orthonormal.
    Invoked in Theorem 9 to justify norm preservation after contracting the left cores.
  • domain assumption FLOP count of economic SVD on an m by n matrix is C_SVD mn min(m,n) with a universal constant C_SVD.
    Used in equations (1), (13), and Algorithm 9 to estimate runtimes and select p; the value of C_SVD does not affect the correctness proof.
  • domain assumption The sparse tensor A is accessible in a format that enumerates nonzero p-fibers efficiently via hash tables or balanced binary search trees.
    Assumed in Section 3.6 for the complexity advantage; it is an implementation detail not needed for correctness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Faster Tensor Train Decomposition for Sparse Data." pith.science (2026). https://pith.science/paper/AZZPNEJS

@misc{pith2026190802721,
  author       = {Pith},
  title        = {Pith review of: Faster Tensor Train Decomposition for Sparse Data},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AZZPNEJS}},
  note         = {Machine review of arXiv:1908.02721}
}
read the original abstract

In recent years, the application of tensors has become more widespread in fields that involve data analytics and numerical computation. Due to the explosive growth of data, low-rank tensor decompositions have become a powerful tool to harness the notorious curse of dimensionality. The main forms of tensor decomposition include CP decomposition, Tucker decomposition, tensor train (TT) decomposition, etc. Each of the existing TT decomposition algorithms, including the TT-SVD and randomized TT-SVD, is successful in the field, but neither can both accurately and efficiently decompose large-scale sparse tensors. Based on previous research, this paper proposes a new quasi-best fast TT decomposition algorithm for large-scale sparse tensors with proven correctness and the upper bound of its complexity is derived. In numerical experiments, we verify that the proposed algorithm can decompose sparse tensors faster than the TT-SVD, and have more speed, precision and versatility than randomized TT-SVD, and it can be used to decomposes arbitrary high-dimensional tensor without losing efficiency when the number of non-zero elements is limited. The new algorithm implements a large-scale sparse matrix TT decomposition that was previously unachievable, enabling tensor decomposition based algorithms to be applied in larger-scale scenarios.

Figures

Figures reproduced from arXiv: 1908.02721 by the authors.

Figure 1
Figure 1. Graphical illustrations of the tensor train (TT) decomposition, where a 3-way tensor [PITH_FULL_IMAGE:figures/full_fig_p004_1.png] view at source ↗
Figure 2
Figure 2. A 1-fiber, a 2-fiber and a 3-fiber of tensor [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗
Figure 3
Figure 3. The nonzero distributions of P(k) , (2 ≤ k ≤ d, k 6= p) and V. 9 [PITH_FULL_IMAGE:figures/full_fig_p009_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The graphical representations of the decomposition forms during the Algorithm 4 [PITH_FULL_IMAGE:figures/full_fig_p011_4.png]
Figure 5
Figure 5. Figure 5: Diagram of matrix-vector-multiplication in the TT-format. [PITH_FULL_IMAGE:figures/full_fig_p018_5.png]
Figure 6
Figure 6. Figure 6: The CPU time and estimated FLOPs with (13) of the FastTT algorithm for different [PITH_FULL_IMAGE:figures/full_fig_p022_6.png]
Figure 7
Figure 7. Figure 7: The sparsity pattern of the coefficient matrix for FDM with 20 [PITH_FULL_IMAGE:figures/full_fig_p023_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 27 canonical work pages

  1. [1]

    The expression of a tensor or a polyadic as a sum of products

    Frank L Hitchcock. The expression of a tensor or a polyadic as a sum of products. Studies in Applied Mathematics , 6(1-4):164–189, 1927

  2. [2]

    Some mathematical notes on three-mode factor analysis

    Ledyard R Tucker. Some mathematical notes on three-mode factor analysis. Psychometrika, 31(3):279–311, 1966

  3. [3]

    Tensor-train decomposition

    Ivan V Oseledets. Tensor-train decomposition. SIAM Journal on Scientific Computing, 33(5):2295–2317, 2011

  4. [4]

    Solution of linear systems and ma- trix inversion in the TT-format

    Ivan V Oseledets and SV Dolgov. Solution of linear systems and ma- trix inversion in the TT-format. SIAM Journal on Scientific Computing , 34(5):A2718–A2739, 2012. 26

  5. [5]

    Enabling high-dimensional hierarchical uncertainty quantifi- cation by ANOVA and tensor-train decomposition

    Zheng Zhang, Xiu Yang, Ivan V Oseledets, George E Karniadakis, and Luca Daniel. Enabling high-dimensional hierarchical uncertainty quantifi- cation by ANOVA and tensor-train decomposition. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems , 34(1):63–76, 2014

  6. [6]

    Model reduction and simu- lation of nonlinear circuits via tensor decomposition

    Haotian Liu, Luca Daniel, and Ngai Wong. Model reduction and simu- lation of nonlinear circuits via tensor decomposition. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 34(7):1059– 1069, 2015

  7. [7]

    Tensor computation: A new framework for high-dimensional problems in EDA

    Zheng Zhang, Kim Batselier, Haotian Liu, Luca Daniel, and Ngai Wong. Tensor computation: A new framework for high-dimensional problems in EDA. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 36(4):521–536, 2017

  8. [8]

    Tensor network alter- nating linear scheme for MIMO Volterra system identification

    Kim Batselier, Zhongming Chen, and Ngai Wong. Tensor network alter- nating linear scheme for MIMO Volterra system identification. Automatica, 84:26–35, 2017

Show all 28 references
  1. [9]

    Approximation of 2 d× 2d matrices using tensor decom- position

    Ivan V Oseledets. Approximation of 2 d× 2d matrices using tensor decom- position. SIAM Journal on Matrix Analysis and Applications , 31(4):2130– 2145, 2010

  2. [10]

    On low-rank approximability of solutions to high-dimensional operator equations and eigenvalue problems

    Daniel Kressner and Andr´ e Uschmajew. On low-rank approximability of solutions to high-dimensional operator equations and eigenvalue problems. Linear Algebra and its Applications , 493:556–572, 2016

  3. [11]

    Computing low-rank approximations of large-scale matrices with the tensor network randomized SVD

    Kim Batselier, Wenjian Yu, Luca Daniel, and Ngai Wong. Computing low-rank approximations of large-scale matrices with the tensor network randomized SVD. SIAM Journal on Matrix Analysis and Applications , 39(3):1221–1244, 2018

  4. [12]

    Efficient low rank tensor ring completion

    Wenqi Wang, Vaneet Aggarwal, and Shuchin Aeron. Efficient low rank tensor ring completion. In IEEE International Conference on Computer Vision (ICCV), pages 5698–5706, 2017

  5. [13]

    Fast and accurate tensor completion with total variation regularized tensor trains

    Ching-Yun Ko, Kim Batselier, Wenjian Yu, and Ngai Wong. Fast and accurate tensor completion with total variation regularized tensor trains. arXiv preprint arXiv:1804.06128 , 2018

  6. [14]

    Tensor train neigh- borhood preserving embedding

    Wenqi Wang, Vaneet Aggarwal, and Shuchin Aeron. Tensor train neigh- borhood preserving embedding. IEEE Transactions on Signal Processing , 66(10):2724–2732, 2018

  7. [15]

    Support vector machine based on low-rank tensor train decomposition for big data applications

    Yongkang Wang, Weicheng Zhang, Zhuliang Yu, Zhenghui Gu, Hao Liu, Zhaoquan Cai, Congjun Wang, and Shihan Gao. Support vector machine based on low-rank tensor train decomposition for big data applications. In 2017 12th IEEE Conference on Industrial Electronics and Applications ...

  8. [16]

    Par- allelized tensor train learning of polynomial classifiers

    Zhongming Chen, Kim Batselier, Johan AK Suykens, and Ngai Wong. Par- allelized tensor train learning of polynomial classifiers. IEEE transactions on neural networks and learning systems , 29(10):4621–4632, 2017

  9. [17]

    Whole brain fMRI pattern analysis based on tensor neural network

    Xiaowen Xu, Qiang Wu, Shuo Wang, Ju Liu, Jiande Sun, and Andrzej Ci- chocki. Whole brain fMRI pattern analysis based on tensor neural network. IEEE Access, 6:29297–29305, 2018

  10. [18]

    TT-cross approximation for mul- tidimensional arrays

    Ivan Oseledets and Eugene Tyrtyshnikov. TT-cross approximation for mul- tidimensional arrays. Linear Algebra and its Applications , 432(1):70–88, 2010

  11. [19]

    Fast adaptive interpolation of multi-dimensional arrays in tensor train format

    Dmitry Savostyanov and Ivan Oseledets. Fast adaptive interpolation of multi-dimensional arrays in tensor train format. In The 2011 International Workshop on Multidimensional (nD) Systems , pages 1–8. IEEE, 2011

  12. [20]

    A randomized tensor train singular value decomposition

    Benjamin Huber, Reinhold Schneider, and Sebastian Wolf. A randomized tensor train singular value decomposition. In Compressed Sensing and its Applications, pages 261–290. Springer, Cham, 2017

  13. [21]

    Finding struc- ture with randomness: Probabilistic algorithms for constructing approxi- mate matrix decompositions

    Nathan Halko, Per-Gunnar Martinsson, and Joel A Tropp. Finding struc- ture with randomness: Probabilistic algorithms for constructing approxi- mate matrix decompositions. SIAM review, 53(2):217–288, 2011

  14. [22]

    A theory of pseudoskeleton approximations

    Sergei A Goreinov, Eugene E Tyrtyshnikov, and Nickolai L Zamarashkin. A theory of pseudoskeleton approximations. Linear algebra and its appli- cations, 261(1-3):1–21, 1997

  15. [23]

    Generic construction of efficient matrix product operators

    C Hubig, IP McCulloch, and U Schollw¨ ock. Generic construction of efficient matrix product operators. Physical Review B, 95(3):035129, 2017

  16. [24]

    Xerus - A general purpose tensor library, 2014–2017

    Benjamin Huber and Sebastian Wolf. Xerus - A general purpose tensor library, 2014–2017

  17. [25]

    Corrected one-site density matrix renormalization group and alternating minimal energy algorithm

    Sergey V Dolgov and Dmitry V Savostyanov. Corrected one-site density matrix renormalization group and alternating minimal energy algorithm. In Numerical Mathematics and Advanced Applications-ENUMATH 2013 , pages 335–343. Springer, 2015

  18. [26]

    SNAP Datasets: Stanford large network dataset collection, June 2015

    Jure Leskovec and Andrej Krevl. SNAP Datasets: Stanford large network dataset collection, June 2015

  19. [27]

    Alan George and Joseph W. Liu. Computer Solution of Large Sparse Pos- itive Definite. Prentice Hall Professional Technical Reference, 1981

  20. [28]

    Introduction to Algorithms

    Thomas H Cormen, Charles E Leiserson, Ronald L Rivest, and Clifford Stein. Introduction to Algorithms. MIT press, 2009. 28

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.