pith. sign in

arxiv: 2605.16610 · v1 · pith:BH4ZYJ5Jnew · submitted 2026-05-15 · 💻 cs.LG · cs.GL

Tensor Cookbook: Mastering Tensors through Diagrams

Beheshteh T. Rakhshan , Guillaume Rabusseau This is my paper

Pith reviewed 2026-05-20 20:05 UTC · model grok-4.3

classification 💻 cs.LG cs.GL
keywords tensor networksdiagrammatic notationtensor algebramachine learningtensor decompositionsgradientshigh-dimensional data
0
0 comments X

The pith

Tensor network diagrams yield shorter and more transparent proofs of tensor identities, rank bounds, and gradient formulas than index manipulation.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper establishes that the graphical language of tensor networks, where contractions appear as edges in a diagram, makes operations on high-dimensional tensors easier to perform and verify. It walks through basic tensor products, reshaping, and decompositions in this notation, then applies the same approach to gradient derivations and probability distributions. A reader would care if the visual method genuinely cuts the length and opacity of algebraic work that grows unwieldy with tensor order. The manuscript positions itself as a self-contained reference to bring this quantum-physics tool into machine learning and numerical analysis.

Core claim

The diagrammatic approach yields genuinely shorter and more transparent proofs of classical identities, rank bounds, and gradient formulas that would otherwise require laborious index manipulation. Operations such as contractions and reshaping are expressed by connecting tensor nodes with edges; classical decompositions and probability manipulations follow directly from the resulting graphs.

What carries the argument

The graphical notation of tensor networks, in which tensors are nodes and index contractions are edges that connect them.

If this is right

  • Gradient formulas for tensor expressions become derivable by diagram rewriting rules instead of index sums.
  • Rank bounds and contraction identities receive visual proofs based on graph connectivity.
  • Tensor decompositions and reshaping operations are expressed directly as diagram transformations.
  • High-dimensional probability distributions can be manipulated by redrawing edges rather than expanding multi-index expressions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same diagrams could be turned into executable tensor-network code for automatic differentiation in libraries.
  • Visual inspection of diagram topology might reveal new low-rank approximations not obvious from index notation.
  • The method may transfer to other graphical calculi used in statistics or signal processing.

Load-bearing premise

That machine learning practitioners can adopt the graphical notation from quantum physics and obtain the claimed simplifications without significant extra background or that the simplifications hold for the examples shown.

What would settle it

A concrete tensor identity or gradient derivation in which the diagrammatic version produces a different or longer result than the standard index calculation.

Figures

Figures reproduced from arXiv: 2605.16610 by Beheshteh T. Rakhshan, Guillaume Rabusseau.

Figure 1
Figure 1. Figure 1: A half-sweep of TT-ALS [PITH_FULL_IMAGE:figures/full_fig_p033_1.png] view at source ↗
read the original abstract

High-dimensional data arise naturally in many areas of science and engineering, including machine learning, signal processing, computational physics, and statistics. Such data are often represented as tensors, multi-dimensional generalizations of matrices. While tensors provide a natural representation for multi-modal structure, their direct manipulation quickly becomes challenging as the order grows: the number of parameters increases exponentially, and algebraic expressions involving many indices become difficult to interpret and implement. Tensor networks (TNs) provide an effective framework for addressing these challenges. Originally introduced by Penrose and developed extensively in quantum physics, the graphical language of tensor networks encodes contractions as edges in a graph, reducing notational overhead and revealing structural properties obscured by index notation. Despite the central role of high-dimensional tensors in modern machine learning and numerical analysis, tensor network diagrams remain underutilized outside quantum computing, partly due to the lack of a self-contained mathematical reference accessible to a broad technical audience. This manuscript provides a self-contained guide to tensor networks and their use in tensor algebra. We present the main operations on tensors, contractions, products, and reshaping through, graphical notation, and show how classical tensor decompositions and related computations are naturally expressed in this framework. We also illustrate how tensor networks simplify the derivation of gradients and the manipulation of high-dimensional probability distributions. Throughout, we show that the diagrammatic approach yields genuinely shorter and more transparent proofs of classical identities, rank bounds, and gradient formulas that would otherwise require laborious index manipulation.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The manuscript is a self-contained tutorial introducing tensor network diagrams for representing and manipulating high-dimensional tensors. It covers standard operations (contractions, products, reshaping), classical decompositions, gradient computations, and high-dimensional probability distributions, with the central claim that the graphical notation from quantum physics yields shorter, more transparent derivations of identities, rank bounds, and gradients than traditional index manipulation.

Significance. If the claimed simplifications are demonstrated with concrete, verifiable examples, the work could provide a useful bridge for machine-learning practitioners to adopt tensor-network techniques, reducing notational barriers for multi-modal data and gradient derivations. The self-contained presentation and focus on ML-relevant applications are strengths, though the absence of direct evidence for brevity claims limits the assessed impact.

major comments (1)
  1. Abstract and main text (gradient and identity sections): The repeated claim that diagrammatic derivations are 'genuinely shorter and more transparent' than index manipulation is not supported by explicit side-by-side comparisons, step counts, or length metrics for any specific identity, rank bound, or gradient formula. Without such evidence the central selling point remains an assertion rather than a demonstrated result.
minor comments (2)
  1. Notation consistency: Ensure that all diagram conventions (e.g., edge labels for indices, node shapes for tensor types) are introduced once with a legend and used uniformly throughout.
  2. References: Add citations to foundational Penrose tensor-network papers and to recent ML applications of tensor networks to strengthen the positioning of the tutorial.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their constructive and insightful report. We appreciate the positive assessment of the manuscript's self-contained nature and relevance to machine learning. We address the single major comment below and will revise the manuscript to incorporate the suggested improvements.

read point-by-point responses

  1. Referee: Abstract and main text (gradient and identity sections): The repeated claim that diagrammatic derivations are 'genuinely shorter and more transparent' than index manipulation is not supported by explicit side-by-side comparisons, step counts, or length metrics for any specific identity, rank bound, or gradient formula. Without such evidence the central selling point remains an assertion rather than a demonstrated result.

    Authors: We agree that the central claim would be substantially strengthened by direct evidence. In the revised version we will add explicit side-by-side comparisons in the abstract and in the sections on identities, rank bounds, and gradients. For at least two representative cases (one identity and one gradient computation) we will present both the traditional index derivation and the diagrammatic derivation, accompanied by a count of algebraic steps or lines of notation. This will allow readers to verify the asserted advantages rather than taking them on faith. revision: yes

Circularity Check

0 steps flagged

Expository guide with no circular derivations or self-referential reductions

full rationale

The manuscript is a self-contained expository guide presenting tensor operations, contractions, decompositions, gradients, and probability distributions via established graphical notation from prior quantum physics literature (Penrose et al.). No load-bearing derivations are shown that reduce by the paper's own equations or self-citations to their inputs by construction; examples illustrate classical identities without redefining them in terms of the diagrammatic output. The claim of shorter proofs is an assertion about pedagogical utility rather than a formal derivation chain that could be circular. The work relies on independent external conventions and does not introduce fitted parameters, ansatzes smuggled via self-citation, or uniqueness theorems from the authors' prior work.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The paper draws on standard tensor algebra and graphical notation conventions established in prior literature, with no new free parameters, ad-hoc axioms, or invented entities introduced.

axioms (1)
  • standard math Standard rules of tensor contraction and index notation as background
    Invoked throughout as the basis for showing graphical equivalents.

pith-pipeline@v0.9.0 · 5791 in / 1081 out tokens · 47715 ms · 2026-05-20T20:05:59.006938+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

48 extracted references · 48 canonical work pages · 5 internal anchors

  1. [1]

    Journal of Statistical Mechanics: Theory and Experiment , volume=

    From density-matrix renormalization group to matrix product states , author=. Journal of Statistical Mechanics: Theory and Experiment , volume=. 2007 , publisher=

    work page 2007

  2. [2]

    Physical review letters , volume=

    Efficient simulation of one-dimensional quantum many-body systems , author=. Physical review letters , volume=. 2004 , publisher=

    work page 2004

  3. [3]

    Numerische Mathematik , volume=

    On manifolds of tensors of fixed TT-rank , author=. Numerische Mathematik , volume=. 2012 , publisher=

    work page 2012

  4. [4]

    Numerical Linear Algebra with Applications , volume=

    On algorithms for and computing with the tensor ring decomposition , author=. Numerical Linear Algebra with Applications , volume=. 2020 , publisher=

    work page 2020

  5. [5]

    SIAM Journal on Matrix Analysis and Applications , volume=

    Tensor ring decomposition: optimization landscape and one-loop convergence of alternating least squares , author=. SIAM Journal on Matrix Analysis and Applications , volume=. 2020 , publisher=

    work page 2020

  6. [6]

    Tensor network contractions for \#

    Biamonte, Jacob D and Morton, Jason and Turner, Jacob , journal=. Tensor network contractions for \#. 2015 , publisher=

    work page 2015

  7. [7]

    SIAM Journal on Computing , volume=

    Simulating quantum computation by contracting tensor networks , author=. SIAM Journal on Computing , volume=. 2008 , publisher=

    work page 2008

  8. [8]

    Journal of Mathematics and Physics , volume=

    The expression of a tensor or a polyadic as a sum of products , author=. Journal of Mathematics and Physics , volume=. 1927 , publisher=

    work page 1927

  9. [9]

    Journal of the ACM (JACM) , volume=

    Most tensor problems are NP-hard , author=. Journal of the ACM (JACM) , volume=. 2013 , publisher=

    work page 2013

  10. [10]

    SIAM review , volume=

    Tensor decompositions and applications , author=. SIAM review , volume=. 2009 , publisher=

    work page 2009

  11. [11]

    Parallel Processing Letters , volume=

    On optimizing a class of multi-dimensional loops with reduction for parallel execution , author=. Parallel Processing Letters , volume=. 1997 , publisher=

    work page 1997

  12. [12]

    Approximation of 2\^

    Oseledets, Ivan V , journal=. Approximation of 2\^. 2010 , publisher=

    work page 2010

  13. [13]

    Combinatorial mathematics and its applications , volume=

    Applications of negative dimensional tensors , author=. Combinatorial mathematics and its applications , volume=

    work page

  14. [14]

    Annals of physics , volume=

    A practical introduction to tensor networks: Matrix product states and projected entangled pair states , author=. Annals of physics , volume=. 2014 , publisher=

    work page 2014

  15. [15]

    Tensor Networks in a Nutshell

    Tensor networks in a nutshell , author=. arXiv preprint arXiv:1708.00006 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  16. [16]

    ACM Transactions on Mathematical Software (TOMS) , volume=

    Algorithm 862: MATLAB tensor classes for fast algorithm prototyping , author=. ACM Transactions on Mathematical Software (TOMS) , volume=. 2006 , publisher=

    work page 2006

  17. [17]

    SIAM Journal on Scientific Computing , volume=

    The alternating linear scheme for tensor optimization in the tensor train format , author=. SIAM Journal on Scientific Computing , volume=. 2012 , publisher=

    work page 2012

  18. [18]

    arXiv preprint arXiv:2202.02138 , year=

    A Practical Guide to the Numerical Implementation of Tensor Networks I: Contractions, Decompositions and Gauge Freedom , author=. arXiv preprint arXiv:2202.02138 , year=

    work page arXiv

  19. [19]

    Physical Review B , volume=

    Gauge fixing, canonical forms, and optimal truncations in tensor networks with closed loops , author=. Physical Review B , volume=. 2018 , publisher=

    work page 2018

  20. [20]

    Psychometrika , volume=

    Some mathematical notes on three-mode factor analysis , author=. Psychometrika , volume=. 1966 , publisher=

    work page 1966

  21. [21]

    Tensor Ring Decomposition

    Tensor ring decomposition , author=. arXiv preprint arXiv:1606.05535 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  22. [22]

    Proceedings of the IEEE International Conference on Computer Vision , pages=

    Efficient low rank tensor ring completion , author=. Proceedings of the IEEE International Conference on Computer Vision , pages=

    work page

  23. [23]

    SIAM Journal on Matrix Analysis and Applications , volume=

    Robust low-rank tensor recovery: Models and algorithms , author=. SIAM Journal on Matrix Analysis and Applications , volume=. 2014 , publisher=

    work page 2014

  24. [24]

    Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages=

    Wide compression: Tensor ring nets , author=. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages=

    work page

  25. [25]

    2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) , pages=

    Higher-dimension tensor completion via low-rank tensor ring decomposition , author=. 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC) , pages=. 2018 , organization=

    work page 2018

  26. [26]

    Multiway data analysis , pages=

    Rank, decomposition, and uniqueness for 3-way and N-way arrays , author=. Multiway data analysis , pages=

    work page

  27. [27]

    Physical Review X , volume=

    Differentiable programming tensor networks , author=. Physical Review X , volume=. 2019 , publisher=

    work page 2019

  28. [28]

    Physical Review E—Statistical, Nonlinear, and Soft Matter Physics , volume=

    Plaquette renormalization scheme for tensor network states , author=. Physical Review E—Statistical, Nonlinear, and Soft Matter Physics , volume=. 2011 , publisher=

    work page 2011

  29. [29]

    , author=

    Multilinear operators for higher-order decompositions. , author=. 2006 , institution=

    work page 2006

  30. [30]

    2005 , publisher=

    Multi-way analysis: applications in the chemical sciences , author=. 2005 , publisher=

    work page 2005

  31. [31]

    Doctoral Thesis, University of Amsterdam , year=

    Multi-way Analysis in the Food Industry, Model, Algorithms and Applications , author=. Doctoral Thesis, University of Amsterdam , year=

    work page

  32. [32]

    SIAM journal on Matrix Analysis and Applications , volume=

    A multilinear singular value decomposition , author=. SIAM journal on Matrix Analysis and Applications , volume=. 2000 , publisher=

    work page 2000

  33. [33]

    Renormalization algorithms for Quantum-Many Body Systems in two and higher dimensions

    Renormalization algorithms for quantum-many body systems in two and higher dimensions , author=. arXiv preprint cond-mat/0407066 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  34. [34]

    SIAM journal on matrix analysis and applications , volume=

    Hierarchical singular value decomposition of tensors , author=. SIAM journal on matrix analysis and applications , volume=. 2010 , publisher=

    work page 2010

  35. [35]

    Journal of Fourier analysis and applications , volume=

    A new scheme for the tensor representation , author=. Journal of Fourier analysis and applications , volume=. 2009 , publisher=

    work page 2009

  36. [36]

    Tensor Network Estimation of Distribution Algorithms

    Tensor Network Estimation of Distribution Algorithms , author=. arXiv preprint arXiv:2412.19780 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  37. [37]

    Advances in neural information processing systems , volume=

    Expressive power of tensor-network factorizations for probabilistic modeling , author=. Advances in neural information processing systems , volume=

    work page

  38. [38]

    Advances in neural information processing systems , volume=

    Tensorizing neural networks , author=. Advances in neural information processing systems , volume=

    work page

  39. [39]

    Physical Review X , volume=

    Unsupervised generative modeling using matrix product states , author=. Physical Review X , volume=. 2018 , publisher=

    work page 2018

  40. [40]

    Biometrika , volume=

    On a formula for the product-moment coefficient of any order of a normal frequency distribution in any number of variables , author=. Biometrika , volume=. 1918 , publisher=

    work page 1918

  41. [41]

    2018 , publisher=

    Matrix variate distributions , author=. 2018 , publisher=

    work page 2018

  42. [42]

    International Conference on Machine Learning , pages=

    Putting MRFs on a tensor train , author=. International Conference on Machine Learning , pages=. 2014 , organization=

    work page 2014

  43. [43]

    The trouble with tensor ring decompositions

    The trouble with tensor ring decompositions , author=. arXiv preprint arXiv:1811.03813 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  44. [44]

    SciPost Physics Codebases , pages=

    The ITensor software library for tensor network calculations , author=. SciPost Physics Codebases , pages=

    work page

  45. [45]

    Journal of Open Source Software , volume=

    quimb: A python package for quantum information and many-body calculations , author=. Journal of Open Source Software , volume=

    work page

  46. [46]

    Journal of Machine Learning Research , volume=

    tntorch: Tensor network learning with PyTorch , author=. Journal of Machine Learning Research , volume=

    work page

  47. [47]

    Advances in Neural Information Processing Systems , volume=

    Convolutions and More as Einsum: A Tensor Network Perspective with Advances for Second-Order Methods , author=. Advances in Neural Information Processing Systems , volume=

    work page

  48. [48]

    Tensor Network: Algorithms, Theory, and Software , howpublished =

    work page