Very Basics of Tensors with Graphical Notations: Unfolding, Calculations, and Decompositions

Tatsuya Yokota

REVIEW 1 major objections 3 minor 35 references

Very Basics of Tensors with Graphical Notations: Unfolding, Calculations, and Decompositions

T0 review · 1 major / 3 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read One graphical notation—a node per tensor, an edge per mode—is shown to cover inner, outer, Hadamard, Kronecker, and Khatri-Rao products, and to make tensor decompositions easier to learn.

desk verdict A genuinely useful graphical tutorial with a real, fixable error: the QR/SVD equations in Sec 6.1/6.2 omit a mode permutation for n<N. read the letter →

arxiv 2411.16094 v1 pith:S7JL7TFW submitted 2024-11-25 cs.LG cs.CVeess.SPstat.ML

Tatsuya Yokota This is my paper · ORCID

classification cs.LGcs.CVeess.SPstat.ML MSC 15A6915A72

keywords tensornetworkdiagramsgraphicalnotationproductsHadamardproductKroneckerKhatri-RaoCPdecompositiontensor-train

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This lecture note sets out to show that a single graphical language, the tensor network diagram, can express most operations used in tensor algebra: inner product, outer product, Hadamard product, Kronecker product, Khatri-Rao product, and mode products. Each tensor is drawn as a node with one edge per mode, and products become edge connections. The claim is that this notation captures the essence of these operations, and that learning tensor decompositions—CP, Tucker, tensor-train—through diagrams is faster and less error-prone than working from formulas alone. The note is a systematic dictionary: it defines each operation symbolically, draws it, and proves the standard identities by rewiring the pictures.

What carries the argument

The load-bearing object is the tensor network diagram: an $N$th-order tensor is a vertex with $N$ emanating edges, a scalar is a vertex with no edges, and contraction—connecting two edges of equal length—is the only operation. Two auxiliary drawn objects carry the argument: the semicircular folding/unfolding node, which represents vectorization and its inverse as a tensor of suitable order, and the super-diagonal tensor (a node whose entries are $1$ only when all indices coincide), which represents the Hadamard product and, by extension, the Khatri-Rao product after unfolding. These pictures turn algebraic identities into edge-rewiring rules; for example, the Kronecker-product identity is shown in one figure by sliding boxes past each other, and the expensive versus cheap contraction order in a tensor network is visible as which intermediate tensor is created.

What would settle it

Write a two-line check in a row-major language: define a 2-by-2 matrix $A = [[1,2],[3,4]]$, let $\mathrm{vec}_{\text{row}}(A) = (1,2,3,4)$, and compute $\text{fold}(\mathrm{vec}_{\text{row}}(A))$ by reshaping back; the resulting matrix differs from $A$. Equivalently, with $G$ the identity, evaluate $\mathrm{vec}(AGB^{\top})$ against $(B \otimes A)\mathrm{vec}(G)$; they disagree unless factors are transposed, showing the drawn identity is convention-bound.

Watch

Extended reading notes

Core claim

The central assertion is that the obstacles a beginner faces in tensor literature—confusing definitions of vectorization, matricization, and the many names of products—are largely notation problems, and that tensor network diagrams dissolve them. Concretely, the note proves that the inner product, outer product, Hadamard product, Kronecker product, Khatri-Rao product, mode product, and tensor product are all special cases of connecting edges between tensor nodes, and that reshaping operations (mode permutation, vectorization, folding, matricization) are linear maps that can themselves be drawn as tensors with a semicircular node. On this foundation, identities such as $(A \otimes B)(C \otimes D)=AC \otimes BD$ and $\mathrm{vec}(AGB^{\top})=(B \otimes A)\mathrm{vec}(G)$ are read directly off the pictures, and every major decomposition—QR, SVD, CP, Tucker, tensor-train, and variants—appears as a small tensor network whose optimization (alternating least squares per factor) follows from the displayed unfolding formulas.

Load-bearing premise

Everything rests on one convention: vectorization uses column-major ordering, so the folding and unfolding diagrams are exact inverses only when the reader's software stores tensors in that same order.

Editorial extensions

If this is right

Every product a reader meets in tensor papers—from vector inner products to multi-tensor contractions—has one uniform drawing, so a beginner can check the meaning of a symbol with a picture.
Reshaping operations are linear and invertible, so any identity stated for a tensor can be restated for its vectorization or matricization without changing the mathematics.
The Hadamard product becomes diagonal-matrix multiplication after vectorization, and its operator is the super-diagonal tensor; this links elementwise operations to linear algebra.
The unfolding formulas for CP and Tucker, such as $X_{(1)} = A\Lambda(C \odot B)^{\top}$, fall out of the same diagrams, which is exactly what an alternating-least-squares implementation needs.
Tensor-train decompositions reduce to chaining third-order cores, and the TT-SVD algorithm is the repeated pattern of SVD on the remaining sub-train, visible in the figures.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

One testable consequence of the note's pedagogical claim: students taught with both the formula and the diagram should make fewer transposition and index-order mistakes than students taught formulas alone; a controlled study could check this.
The column-major linearization is a convention, not a mathematical necessity. Any software that uses row-major storage must transpose the factors in $\mathrm{vec}(AGB^{\top})=(B \otimes A)\mathrm{vec}(G)$; readers porting the diagrams to such software should treat the drawn identities as holding after that adjustment.
The same graphical dictionary can be carried further than the note goes: any contraction pattern—including hierarchical Tucker, tensor rings, and fully-connected networks—is already representable, and the note's rules for contraction cost apply directly to those variants.
A library that implements these diagrams as a typing system (each edge carries its length) could verify whether user-defined tensor formulas match an identity or whether an implicit transpose is hiding.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things you should know. First, the paper is a genuinely useful graphical tutorial for tensor basics; the operator-as-tensor framing and the diagrammatic proofs are well done. Second, the stress-test concern is correct: the QR/SVD equations in Sec 6.1/6.2 are wrong for n

What the paper does well: it fills a pedagogical gap by collecting standard tensor material (CP, Tucker, TT, HOSVD) and expressing it in tensor network diagrams. The sections on folding/unfolding operators and Hadamard product operators as tensors are a nice unifying idea. The graphical proof of (A⊗B)(C⊗D)=AC⊗BD and the vectorization identity vec(AGB^T)=(B⊗A)vec(G) are clear and genuinely helpful. The exposition is honest about scope: it says optimization details are omitted, and it does not oversell novelty.

Soft spots, in proportion. The QR/SVD error is load-bearing because it teaches students an incorrect equation; it appears in the text and would mislead anyone who tries to reproduce the factors. Fixing it is straightforward: add an explicit mode permutation or define the product to insert the contracted mode in its original position. There are also minor typos ('entrie', 'all orghogonality') and the n×m product notation, while useful, needs to be applied with care. The column-major vectorization convention is not an error; it is stated explicitly, though readers coming from row-major software will need to transpose.

Who this is for: newcomers to tensor decompositions and tensor networks, and instructors looking for a diagram-first introduction. It is a lecture note, not a research contribution, and it should be judged as such. With the mode-order fix, it would be a solid tutorial worth recommending. As written, the QR/SVD section should not be cited or taught without correction.

Recommendation: engage with it, but send it back for revision. The paper deserves serious peer review because the pedagogical value is real and the flaw is local and fixable. I would not cite it in its current form.

Referee Report

1 major / 3 minor

Summary. This lecture note proposes tensor network (Penrose) diagrams as a pedagogical notation for basic tensor operations and decompositions. It defines tensors, modes, fibers, slices, and reshaping operations (mode permutation, vectorization, matricization, folding), then presents graphical descriptions of Hadamard, inner, outer, Kronecker, Khatri-Rao, mode, and tensor products. It concludes with QR, SVD, CP, Tucker, TT, and related decompositions, using diagrams to illustrate standard identities and algorithms such as TT-SVD. The paper's intended contribution is expository: to lower the entry barrier to tensor decomposition literature by making the building blocks visually explicit.

Significance. The manuscript is a well-organized tutorial rather than a research contribution, and its value rests on correctness and clarity. Its strengths are the consistent use of diagrams to prove standard identities (e.g., (A⊗B)(C⊗D)=AC⊗BD in Fig. 30, vec(AGB^T)=(B⊗A)vec(G) in Fig. 31, and (A⊙B)^T(A⊙B)=A^T A ⊙ B^T B in Fig. 33) and the explicit treatment of vectorization and unfolding conventions. Under the column-major convention fixed in Eq. (24), most checked identities are consistent. However, the tensor QR/SVD statements in Section 6 contain a mode-order error that affects the correctness of the central decomposition diagrams; once corrected, the note would be a useful reference for students and practitioners.

major comments (1)

[Section 6.1, Eq. (201); Section 6.2, Eq. (212); Figs. 43–44] The claimed equivalence X^T_(n) = Q^T_(n) R ⇔ X = Q n×1 R is false for n<N under the tensor-product definition of Sec. 5.13. That definition contracts the n-th mode of Q with the first mode of R and concatenates the remaining modes of Q followed by the remaining mode of R, so Q n×1 R has mode order (1,...,n−1,n+1,...,N,n). In contrast, X has mode order (1,...,N). For example, when N=3 and n=1, Q 1×1 R equals permute_{[2,3,1]}(X), not X. The same issue occurs in Eq. (212), where U n×1 ΣV^T also produces a permuted version of X for n<N. The fix is to include the inverse mode permutation (e.g., X = permute^{-1}_{[1,...,n-1,n+1,...,N,n]}(Q n×1 R)) or to introduce a mode-inserting tensor product; without this correction, the graphical QR and SVD representations misstate the mode order of the reconstructed tensor.

minor comments (3)

[Section 4.2, Eq. (24)] The recursive definition of vec relies on the column-major convention. Please add an explicit statement that all reshaping identities are convention-dependent and that readers using row-major software must apply transposition adjustments; otherwise the pedagogical promise of the diagrams is weakened for a large part of the intended audience.
[Section 5.13] The general tensor-product definition should specify the output mode order explicitly for the case of multiple contraction pairs and for the zero-contraction case in Eq. (186); the current notation is understandable from the examples but not fully formal.
[Various] There are several typographical errors that should be corrected: 'entrie' at Eq. (111), 'vectorrization' in Section 4.2, 'representad' after Eq. (171), 'all orghogonality' in Section 6.4, and 'prduct' in the caption of Figure 38.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: standard tensor algebra exposition; sole self-citation is terminological and not load-bearing.

full rationale

This lecture note is a self-contained exposition of established tensor definitions and identities. The central content is built forward from explicit entrywise definitions (vectorization in Sec. 4.2, mode product in Sec. 5.12, tensor product in Sec. 5.13), and the displayed identities are derived from those definitions by linearity and contraction, for example Eqs. (142), (151), (172)-(174), (217)-(220), and (223)-(226). No parameter is fitted to data and no quantity is predicted from its own definition. The only self-citation is [18] (Matsui and Yokota) in Sec. 5.4, where the alignability condition for broadcasting is given a name; this reference is terminological and does not support any of the paper's proofs or decompositions. No uniqueness theorem from the authors' prior work is invoked, and no ansatz is imported via citation. The suspicious identities in Secs. 6.1-6.2 (Eqs. (200)-(201) and (211)-(212)) involve a mode-order mismatch: with the tensor product defined in Sec. 5.13, Q n x 1 R has the contracted R-mode appended after the remaining modes of Q, so for n<N it equals a mode permutation of X rather than X itself. This is an internal mathematical correctness issue, not a circularity: X is not defined as the right-hand side, nor is the formula fitted from X. The derivation chain is therefore self-contained; the identities can be checked against standard tensor algebra without reference to the paper's conclusions.

Assumptions & free parameters 0 free parameters · 6 assumptions · 0 invented entities

The note introduces no free parameters and no new entities. It relies on standard mathematical facts and previously published tensor decomposition models; the only self-referential input is the broadcasting definition from the author's own paper [18], which is used as a formal definition in Sec. 5.4.

assumptions (6)

domain assumption Existence of SVD for any matrix
Used in Sec. 6.2 to define matrix SVD and tensor SVD via matricization.
domain assumption Eckart-Young theorem gives the optimal rank-K approximation
Stated in Sec. 6.2 and cited to [9] without proof.
domain assumption Existence of QR decomposition for any tall matrix
Used in Sec. 6.1 and Sec. 6.5 for orthogonalization of tensor decompositions.
standard math Standard linear algebra identities for Kronecker and Khatri-Rao products
Many identities in Sec. 5.10 and Sec. 5.11 are stated as facts; these are standard and proved in textbooks.
domain assumption Validity of the graphical notation as a calculus for contractions
The whole note assumes that the diagrammatic representation corresponds exactly to tensor contractions; this is a standard convention in the tensor network literature.
domain assumption CP, Tucker, TT, TR, and FCTN decompositions are well-defined as in the cited literature
Sec. 6.3 through Sec. 6.6 introduce these models by citation; their properties are not proven in the note.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Very Basics of Tensors with Graphical Notations: Unfolding, Calculations, and Decompositions." pith.science (2026). https://pith.science/paper/S7JL7TFW

@misc{pith2026241116094,
  author       = {Pith},
  title        = {Pith review of: Very Basics of Tensors with Graphical Notations: Unfolding, Calculations, and Decompositions},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/S7JL7TFW}},
  note         = {Machine review of arXiv:2411.16094}
}

read the original abstract

Tensor network diagram (graphical notation) is a useful tool that graphically represents multiplications between multiple tensors using nodes and edges. Using the graphical notation, complex multiplications between tensors can be described simply and intuitively, and it also helps to understand the essence of tensor products. In fact, most of matrix/tensor products including inner product, outer product, Hadamard product, Kronecker product, and Khatri-Rao product can be written in graphical notation. These matrix/tensor operations are essential building blocks for the use of matrix/tensor decompositions in signal processing and machine learning. The purpose of this lecture note is to learn the very basics of tensors and how to represent them in mathematical symbols and graphical notation. Many papers using tensors omit these detailed definitions and explanations, which can be difficult for the reader. I hope this note will be of help to such readers.

Figures

Figures reproduced from arXiv: 2411.16094 by the authors.

**Figure 2.** Tensors ・・・・・・・・・・・・・・・・・・・・・・・・ block matrix of matrices block vector of tensors ・・・・・・・・・・・・・・・・・・・・・・・・・・・ block tensor of vectors [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Different views of fourth-order tensors vector（1st order tensor） vector of vectors（2nd order tensor） vector of vectors of vectors（3rd order tensor） vector of vectors of vectors of vectors（4th order tensor） [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

Figures from the paper (49 more)

**Figure 4.** Figure 4: Hierarchical views of tensors 2.3 Tensors Tensors are generalizations of vectors and matrices. Vectors are first-order tensors and matrices are second-order tensors1 . A third-order tensor A ∈ R I×J×K is a cubic array which K matrices {A1, A2, ..., AK} arranged in dept…

**Figure 5.** Figure 5: Modes for tensors. of the tensor A [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

**Figure 6.** Figure 6: Tensors in graphical notation. and the second mode being unimportant. There is no meaning other than the symbols to distinguish each one. 2.5 Tensor network diagrams In [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

**Figure 7.** Figure 7: Entry of a third-order tensor. input output one-hot encode inner product [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

**Figure 9.** Figure 9: Fibers of third-order tensor (column, row, and tube). [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]

**Figure 10.** Figure 10: Slices of third-order tensor. Fiber as a function As in the discussion of entries, fibers can be thought of as functions that take an index as input and output a vector. For example, a fiber along the first mode takes (j, k) as input and outputs the vector A(:, j, k),…

**Figure 11.** Figure 11: Image of matrix transposition. mode permutation of a matrix mode permutation of a tensor [PITH_FULL_IMAGE:figures/full_fig_p016_11.png]

**Figure 12.** Figure 12: Image of mode permutation of tensor. Inverse of mode permutation The inverse operation of the tensor Y to the original tensor X is written as X = permute−1 p (Y). For example, when performing mode permutation with [2, 3, 1] for third-order tensor X ∈ R I×J×K, it is ex…

**Figure 13.** Figure 13: Vectorization of a matrix and a tensor. matrices makes a third-order tensor, and arranging third-order tensors makes a fourth-order tensor · · · . We also assumed that the “arrangement directions” are different at each stage. So what happens if they are all in the sam…

**Figure 14.** Figure 14: Entries of a tensor and its vectorization. [PITH_FULL_IMAGE:figures/full_fig_p018_14.png]

**Figure 15.** Figure 15: Vectorization of tensors. Using this, the vectorization of an Nth-order tensor (2 < N) can be defined as follows: vec(T ) = vec([vec(T 1), ..., vec(T IN )]) ∈ R QN n=1 In . (24) Although it is not written explicitly, Eq. (24) defines the vectorization of an Nth-order …

**Figure 16.** Figure 16: Matricization of a third-order tensor. order tensor into a matrix. In particular, mode-n matricization is an operation of selecting the n-th mode as the first mode and combining all the remaining modes into the second mode to a matrix. For example, there are three pat…

**Figure 17.** Figure 17: k-unfolding. The mode-n matricization for an Nth-order tensor T ∈ R I1×I2×···×IN can also be written in general as follows: T(n) =       vec(PnT (1, :, ..., :))⊤ vec(PnT (2, :, ..., :))⊤ . . . vec(PnT (In, :, ..., :))⊤       ∈ R In× Q k̸=n Ik , (32) PnT = …

**Figure 18.** Figure 18: Folding and unfolding. 4.4 Folding (tensorization) Up to this point, we have introduced reshaping of tensors into vectors and matrices. These operations are the transformation from higher-order tensors into lower-order tensors, and are called unfolding in general. In …

**Figure 19.** Figure 19: Inverse of folding and unfolding. and it is the inverse transformation of vectorization as fold(I,J,K) (vec(A)) = A, (43) vec(fold(I,J,K) (a)) = a. (44) [PITH_FULL_IMAGE:figures/full_fig_p023_19.png]

**Figure 20.** Figure 20: Tensor representations of folding and unfolding operators. [PITH_FULL_IMAGE:figures/full_fig_p024_20.png]

**Figure 21.** Figure 21: Hadamard products. 5.2 Entry-wise multiplication and division Hadamard product Entry-wise multiplication of two tensors is called the Hadamard product11. In this note, the Hadamard product is represented as 12 . First, let us consider the case of matrices. The Hadama…

**Figure 22.** Figure 22: Super-diagonal tensors and their properties. Mode product of a [PITH_FULL_IMAGE:figures/full_fig_p029_22.png]

**Figure 23.** Figure 23: Inner products. where {x1, ..., xJ } are data, and µ = 1 J PJ j=1 xj is a mean vector. For matrix representation of data X = [x1, ..., xJ ], its centralization Xe = [x˜1, ..., x˜J ] can be written by Xe = X ⊟ µ = X − [µ, ..., µ], (98) where µ is broadcasted along row …

**Figure 24.** Figure 24: Trace. results a scalar value which is represented as a node without edges in the diagram. Such operations (computations) are called tensor network contractions. Contraction is usually used in the context of operations that actually spend the computational costs to e…

**Figure 25.** Figure 25: Outer products. The outer product of three vectors results in a third-order tensor X = a◦b◦c ∈ R I×J×K, and each entry is given by Xijk = aibjck. (109) In general, the outer product of N vectors a1 ∈ R I1 , · · · , aN ∈ R IN is denoted by a1 ◦ · · · ◦ aN . (110) The o…

**Figure 26.** Figure 26: Sum of all entries [PITH_FULL_IMAGE:figures/full_fig_p034_26.png]

**Figure 28.** Figure 28: Square of Frobenius norm. Each entry of Y is an inner product. The another one is as follow: Y = AX =    | | a1 · · · aP | |        — x˜ ⊤ 1 — . . . — x˜ ⊤ P —     = a1 ◦ x˜1 + · · · + aP ◦ x˜P . (117) Y is given in the form of a sum of outer products. T…

**Figure 29.** Figure 29: Kronecker product. 5.10 Kronecker product Kronecker product is defined for two matrices A ∈ R I×J and B ∈ R P ×Q and denoted as A ⊗ B. (121) When the size of A is (I, J) and the size of B is (P, Q), then the size of their Kronecker product A ⊗ B is (P I, QJ). It can b…

**Figure 30.** Figure 30: Proof of (A ⊗ B)(C ⊗ D) = AC ⊗ BD. It’s a little complicated, but we have Y = permute[2,4,1,3](X ), (126) X = fold(P,I,Q,J) (A ⊗ B) ∈ R P ×I×Q×J , (127) Y = fold(I,J,P,Q) (vec(A) ◦ vec(B)) ∈ R I×J×P ×Q. (128) This suggests a relationship between the outer product and …

**Figure 31.** Figure 31: Proof of vec(AGB⊤) = (B ⊗ A)vec(G). Multiplying (A−1 ⊗ B−1 ) on the right also results in an identity matrix (A ⊗ B)(A−1 ⊗ B −1 ) = AA−1 ⊗ BB−1 = I ⊗ I = I, (139) then (A−1⊗B−1 ) is the inverse matrix of (A⊗B). The case of pseudo-inverse can be proved in a similar way…

**Figure 32.** Figure 32: Khatri-Rao product. 5.11 Khatri-Rao product Khatri-Rao product is defined for two matrices of which the same row length A = [a1, ..., aR] ∈ R I×R，B = [b1, ..., bR] ∈ R J×R and given by A ⊙ B = [a1 ⊗ b1, ..., aR ⊗ bR] ∈ R JI×R. (145) The Khatri-Rao product is an operat…

**Figure 33.** Figure 33: Proof of (A⊙B) ⊤(A⊙B) = A⊤AB⊤B and ⟨ar⊗br, ar ′ ⊗br ′⟩ = ⟨ar, ar ′⟩⟨br, br ′⟩. Although (151) is not straightforward, it can by verified by focusing on (r, r′ )- th entry as follow: ⟨ar ⊗ br, ar ′ ⊗ br ′⟩ = X I i=1 X J j=1 ar(i)br(j)ar ′(i)br ′(j) = X I i=1 X J j=1 a…

**Figure 34.** Figure 34: Mode product X ×1 A [PITH_FULL_IMAGE:figures/full_fig_p041_34.png]

**Figure 36.** Figure 36: Diagrams of X ×1 A ×2 B and (X ×1 A) ×1 A′ . Regarding (162), this is similar to the fact that for the matrix product of three matrices, AGB = A(GB) = (AG)B holds. The order does not matter when the mode products are performed from different modes. However, the order…

**Figure 37.** Figure 37: Vectorization and matricization of all-mode product [PITH_FULL_IMAGE:figures/full_fig_p043_37.png]

**Figure 38.** Figure 38: Examples of tensor prduct. More examples are as follows: (R, L, I)-tensor 3× 1 (I, J, K)-tensor = (R, L, J, K)-tensor, (184) (I, J, L)-tensor 1,2× 3,2 (R, J, I)-tensor = (L, R)-matrix, (185) (I, J)-matrix × (K, L, R)-tensor = (I, J, K, L, R)-tensor. (186) In (184), th…

**Figure 39.** Figure 39: TT product. Tensor-train (TT) product Tensor-train (TT) product is an operation that connects the final mode of the first operand tensor X with the first mode of the second operand tensor Y and it is denoted as ⟨⟨X , Y⟩⟩. (193) In case of that the TT product of a (M +…

**Figure 40.** Figure 40: A tensor network and its evaluation [PITH_FULL_IMAGE:figures/full_fig_p047_40.png]

**Figure 41.** Figure 41: A step-by-step procedure for evaluating a tensor network. The two [PITH_FULL_IMAGE:figures/full_fig_p047_41.png]

**Figure 42.** Figure 42: Computational cost for evaluating a TN. 48 [PITH_FULL_IMAGE:figures/full_fig_p048_42.png]

**Figure 43.** Figure 43: QR decompositions of a matrix and a tensor. [PITH_FULL_IMAGE:figures/full_fig_p050_43.png]

**Figure 44.** Figure 44: SVD and truncated SVD of a tall matrix ( [PITH_FULL_IMAGE:figures/full_fig_p051_44.png]

**Figure 45.** Figure 45: Various forms of SVD for a tensor. and this fact is known as the Eckart-Young theorem [9]. The discarded components by this approximation are given as X − Yb = X R r ′=K+1 σr ′ur ′ ◦ vr ′. (209) Let U˜ = [u1, ..., vK], V˜ = [v1, ..., vK], and Σ˜ = diag(σ1, ..., σK), …

**Figure 46.** Figure 46: CP decomposition and its unfolding. Let D ∈ R R×R×R be a super-diagonal tensor with diagonal entries λ1, ..., λR as Dpqr = ( λr (p = q = r) 0 otherwise , (214) and A = [a1, ..., aR] ∈ R I×R, B = [b1, ..., bR] ∈ R J×R, and C = [c1, ..., cR] ∈ R K×R, then the CP decompo…

**Figure 47.** Figure 47: Tucker decompositions. Unfolding of TKD Since Tucker decomposition is mathematically equivalent to the all-mode product, its unfolding is also the same as shown in [PITH_FULL_IMAGE:figures/full_fig_p055_47.png]

**Figure 48.** Figure 48: Orthogonalization of TKD [PITH_FULL_IMAGE:figures/full_fig_p056_48.png]

**Figure 50.** Figure 50: Process for obtaining HOSVD. where U ∈ R I×I : left singular vectors of X(1), (233) V ∈ R J×J : left singular vectors of X(2), (234) W ∈ R K×K : left singular vectors of X(3). (235) From (232), we can see this is a type of Tucker decomposition. Since the factor matric…

**Figure 51.** Figure 51: TT decomposition. Truncated HOSVD Since the eigenvalues are arranged in descending order, we can expect that the absolute value of each element of the core tensor (the magnitude of the contribution of the (i, j, k) rank 1 component) decreases from H111 to HIJK. From …

**Figure 52.** Figure 52: Orthogonalization of TT decomposition using QR decomposition. [PITH_FULL_IMAGE:figures/full_fig_p060_52.png]

**Figure 53.** Figure 53: Multiple ways of orthogonalizing TT decompositions. [PITH_FULL_IMAGE:figures/full_fig_p060_53.png]

**Figure 54.** Figure 54: Frobenius norm of orthogonalized TT decomposition. [PITH_FULL_IMAGE:figures/full_fig_p061_54.png]

**Figure 56.** Figure 56: Process of TT-SVD. TT-SVD Tensor-train singular value decomposition (TT-SVD) [22] is an algorithm to obtain TT decomposition of a tensor A ∈ R I1×I2×···×IN as A = ⟨⟨G1,G2, ...,GN ⟩⟩. (269) To achieve (269), we set the TT rank by Rk ← rank(A⟨k⟩ ). (270) [PITH_FULL_IMA…

**Figure 57.** Figure 57: Tensor networks. and C is obtained as the remaining tensor. In a similar way, G3, ..., GN can be sequentially obtained by using SVD. When Rk is set to less than rank(A⟨k⟩ ) and replace SVD with truncated SVD with rank Rk, an error will occur and the result is called t…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

35 extracted references · 31 canonical work pages

[1]

Amiridi, N

M. Amiridi, N. Kargas, and N. D. Sidiropoulos. Low-rank characteristic tensor density estimation part I: Foundations. IEEE Transactions on Signal Processing, 70:2654–2668, 2022. 3

work page 2022
[2]

Amiridi, N

M. Amiridi, N. Kargas, and N. D. Sidiropoulos. Low-rank characteristic tensor density estimation part II: Compression and latent density esti- mation. IEEE Transactions on Signal Processing , 70:2669–2680, 2022. 3

work page 2022
[3]

Eckart-Young

J. D. Carroll and J.-J. Chang. Analysis of individual differences in mul- tidimensional scaling via an N-way generalization of “Eckart-Young” de- composition. Psychometrika, 35(3):283–319, 1970. 3, 52

work page 1970
[4]

Cichocki, N

A. Cichocki, N. Lee, I. Oseledets, A.-H. Phan, Q. Zhao, and D. P. Mandic. Tensor networks for dimensionality reduction and large-scale optimiza- tion: Part 1 low-rank tensor decompositions. Foundations and Trends® in Machine Learning , 9(4-5):249–429, 2016. 3

work page 2016
[5]

Cichocki, D

A. Cichocki, D. Mandic, L. De Lathauwer, G. Zhou, Q. Zhao, C. Caiafa, and H. A. Phan. Tensor decompositions for signal processing applications: From two-way to multiway component analysis. IEEE Signal Processing Magazine, 32(2):145–163, 2015. 3

work page 2015
[6]

Cichocki, A.-H

A. Cichocki, A.-H. Phan, Q. Zhao, N. Lee, I. Oseledets, M. Sugiyama, and D. P. Mandic. Tensor networks for dimensionality reduction and large- scale optimization: Part 2 applications and future perspectives. Founda- tions and Trends® in Machine Learning , 9(6):431–673, 2017. 3

work page 2017
[7]

Cichocki, R

A. Cichocki, R. Zdunek, A. H. Phan, and S.-I. Amari. Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-Way Data Analysis and Blind Source Separation. John Wiley & Sons, Ltd, 2009. 26

work page 2009
[8]

De Lathauwer, B

L. De Lathauwer, B. De Moor, and J. Vandewalle. A multilinear singular value decomposition. SIAM Journal on Matrix Analysis and Applications, 21(4):1253–1278, 2000. 56 65

work page 2000

Show all 35 references

[9]

Eckart and G

C. Eckart and G. Young. The approximation of one matrix by another of lower rank. Psychometrika, 1(3):211–218, 1936. 52

1936
[10]

Y. N. Fern´ andez, M. K. Ritter, M. Jeannin, J.-W. Li, T. Kloss, T. Louvet, S. Terasaki, O. Parcollet, J. von Delft, H. Shinaoka, and X. Waintal. Learning tensor networks with tensor cross interpolation: new algorithms and libraries. arXiv preprint arXiv:2407.02454 , 2024. 3, 10

2024 arXiv
[11]

Grasedyck

L. Grasedyck. Hierarchical singular value decomposition of tensors. SIAM Journal on Matrix Analysis and Applications , 31(4):2029–2054, 2010. 63

2010
[12]

explanatory

R. A. Harshman. Foundations of the PARAF AC procedure: Models and conditions for an “explanatory” multimodal factor analysis. UCLA Work- ing Paper in Phonetics , 16:1–84, 1970. 3, 52

1970
[13]

F. L. Hitchcock. The expression of a tensor or a polyadic as a sum of products. Journal of Mathematics and Physics , 6(1-4):164–189, 1927. 52

1927
[14]

Holtz, T

S. Holtz, T. Rohwedder, and R. Schneider. The alternating linear scheme for tensor optimization in the tensor train format. SIAM Journal on Scientific Computing , 34(2):A683–A713, 2012. 3

2012
[15]

R. A. Horn and C. R. Johnson. Matrix Analysis . Cambridge University Press, second edition, 2012. 49

2012
[16]

A. K. Jain. Fundamentals of Digital Image Processing . Prentice-Hall,
[17]

T. G. Kolda and B. W. Bader. Tensor decompositions and applications. SIAM Review, 51(3):455–500, 2009. 3, 26

2009
[18]

Matsui and T

Y. Matsui and T. Yokota. Broadcast product: Shape-aligned element- wise multiplication and beyond. arXiv preprint arXiv:2409.17502 , 2024. 4, 30

2024 arXiv
[19]

Memmel, C

E. Memmel, C. Menzen, J. Schuurmans, F. Wesel, and K. Batselier. Po- sition: Tensor networks are a valuable asset for green AI. In Proceedings of ICML , 2024. 3

2024
[20]

Novikov, D

A. Novikov, D. Podoprikhin, A. Osokin, and D. P. Vetrov. Tensorizing neural networks. Proceedings of NeurIPS, 28, 2015. 3 66

2015
[21]

G. S. Novikov, M. E. Panov, and I. V. Oseledets. Tensor-train density estimation. In Proceedings of Conference on Uncertainty in Artificial Intelligence, pages 1321–1331. PMLR, 2021. 3, 12

2021
[22]

I. V. Oseledets. Tensor-train decomposition. SIAM Journal on Scientific Computing, 33(5):2295–2317, 2011. 3, 58, 62

2011
[23]

R. Penrose. Applications of negative dimensional tensors. Combinatorial Mathematics and its Applications , 1:221–244, 1971. 3, 10

1971
[24]

S.-J. Ran, E. Tirrito, C. Peng, X. Chen, L. Tagliacozzo, G. Su, and M. Lewenstein. Tensor Network Contractions: Methods and Applications to Quantum Many-Body Systems . Springer Nature, 2020. 3, 58

2020
[25]

Sengupta, S

R. Sengupta, S. Adhikary, I. Oseledets, and J. Biamonte. Tensor networks in machine learning. European Mathematical Society Magazine, (126):4– 12, 2022. 3

2022
[26]

N. D. Sidiropoulos, L. De Lathauwer, X. Fu, K. Huang, E. E. Papalex- akis, and C. Faloutsos. Tensor decomposition for signal processing and machine learning. IEEE Transactions on Signal Processing, 65(13):3551– 3582, 2017. 3

2017
[27]

Stoudenmire and D

E. Stoudenmire and D. J. Schwab. Supervised learning with tensor net- works. Proceedings of NeurIPS, 29, 2016. 3, 12

2016
[28]

G. Strang. Linear Algebra and Learning from Data. Wellesley-Cambridge Press Cambridge, 2019. 26

2019
[29]

J. K. Taylor. An introduction to graphical tensor notation for mechanistic interpretability. arXiv preprint arXiv:2402.01790 , 2024. 3, 10

2024 arXiv
[30]

L. R. Tucker. Implications of factor analysis of three-way matrices for measurement of change. Problems in Measuring Change , pages 122–137,
[31]

L. R. Tucker. The extension of factor analysis to three-dimensional ma- trices. Contributions to Mathematical Psychology , pages 109–127, 1964. 54

1964
[32]

L. R. Tucker. Some mathematical notes on three-mode factor analysis. Psychometrika, 31(3):279–311, 1966. 54 67

1966
[33]

T. Xiang. Density Matrix and Tensor Network Renormalization . Cam- bridge University Press, 2023. 58

2023
[34]

Q. Zhao, G. Zhou, S. Xie, L. Zhang, and A. Cichocki. Tensor ring de- composition. arXiv preprint arXiv:1606.05535 , 2016. 3, 63

2016 arXiv
[35]

Zheng, T.-Z

Y.-B. Zheng, T.-Z. Huang, X.-L. Zhao, Q. Zhao, and T.-X. Jiang. Fully- connected tensor network decomposition and its application to higher- order tensor completion. In Proceedings of AAAI Conference on Artificial Intelligence, pages 11071–11078, 2021. 64 68

2021

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [1]

Amiridi, N

M. Amiridi, N. Kargas, and N. D. Sidiropoulos. Low-rank characteristic tensor density estimation part I: Foundations. IEEE Transactions on Signal Processing, 70:2654–2668, 2022. 3

work page 2022

[2] [2]

Amiridi, N

M. Amiridi, N. Kargas, and N. D. Sidiropoulos. Low-rank characteristic tensor density estimation part II: Compression and latent density esti- mation. IEEE Transactions on Signal Processing , 70:2669–2680, 2022. 3

work page 2022

[3] [3]

Eckart-Young

J. D. Carroll and J.-J. Chang. Analysis of individual differences in mul- tidimensional scaling via an N-way generalization of “Eckart-Young” de- composition. Psychometrika, 35(3):283–319, 1970. 3, 52

work page 1970

[4] [4]

Cichocki, N

A. Cichocki, N. Lee, I. Oseledets, A.-H. Phan, Q. Zhao, and D. P. Mandic. Tensor networks for dimensionality reduction and large-scale optimiza- tion: Part 1 low-rank tensor decompositions. Foundations and Trends® in Machine Learning , 9(4-5):249–429, 2016. 3

work page 2016

[5] [5]

Cichocki, D

A. Cichocki, D. Mandic, L. De Lathauwer, G. Zhou, Q. Zhao, C. Caiafa, and H. A. Phan. Tensor decompositions for signal processing applications: From two-way to multiway component analysis. IEEE Signal Processing Magazine, 32(2):145–163, 2015. 3

work page 2015

[6] [6]

Cichocki, A.-H

A. Cichocki, A.-H. Phan, Q. Zhao, N. Lee, I. Oseledets, M. Sugiyama, and D. P. Mandic. Tensor networks for dimensionality reduction and large- scale optimization: Part 2 applications and future perspectives. Founda- tions and Trends® in Machine Learning , 9(6):431–673, 2017. 3

work page 2017

[7] [7]

Cichocki, R

A. Cichocki, R. Zdunek, A. H. Phan, and S.-I. Amari. Nonnegative Matrix and Tensor Factorizations: Applications to Exploratory Multi-Way Data Analysis and Blind Source Separation. John Wiley & Sons, Ltd, 2009. 26

work page 2009

[8] [8]

De Lathauwer, B

L. De Lathauwer, B. De Moor, and J. Vandewalle. A multilinear singular value decomposition. SIAM Journal on Matrix Analysis and Applications, 21(4):1253–1278, 2000. 56 65

work page 2000