REVIEW 4 major objections 5 minor 13 references
Locally orderless tensor networks for classifying two- and three-dimensional medical images
T0 review · 4 major / 5 minor · reviewed 2026-08-27 · deepseek-v4-flash
Pith's one-line read A tensor-network classifier, LoTeNet, treats small image patches as orderless, contracts them hierarchically with matrix-product-state blocks, and reports accuracy comparable to CNN baselines on 2D and 3D medical images while using a…
desk verdict Genuine architectural advance for MPS-based image classification with convincing memory savings, but the performance claims need matched comparisons and error bars before they are accepted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the locally orderless tensor network (LoTeNet): a stack of matrix-product-state (MPS) blocks separated by squeeze operations. The squeeze operation $\psi(\cdot;k)$ reshapes each $k\times\cdots\times k$ patch into a vector whose feature dimension is $C\cdot k^S$, moving local spatial structure into the feature dimension before any tensor contraction. Each MPS block approximates a high-dimensional linear weight tensor as a chain of low-order tensors of bond dimension $\beta$, and contracting it with a squeezed patch produces a $\nu=\beta$-dimensional summary. Reassembling those summaries into a lower-resolution image and repeating the process builds a hierarchy that retains global structure without ever flattening the whole image. The design lowers the contraction cost from $O(N d \beta^2)$ for a single-layer tensor network to roughly $O\!\left(\frac{N}{k^{2L}}\, L k^2 d \beta^2\right)$, and it avoids materializing large intermediate feature maps, which is why GPU memory stays low.
What would settle it
On the LIDC task, train a linear SVM or logistic regression on the same squeezed local patch features that feed LoTeNet. If that linear model reaches AUC near LoTeNet's 0.874, the MPS hierarchy adds little; if it is far below, the hierarchy is load-bearing. Independently, measure LoTeNet's peak GPU memory while doubling the batch size: if memory grows linearly with the number of patches rather than remaining nearly flat, the claimed source of the memory advantage fails.
Extended reading notes
Core claim
LoTeNet's central claim is that a hierarchy of linear tensor contractions, with no nonlinear activation functions, can classify high-resolution medical images on par with or better than convolutional baselines. The input is cut into non-overlapping patches of side length $k$; a squeeze operation flattens each patch's spatial positions into the feature dimension, giving each local region a feature map of size $C\cdot k^S$. A separate MPS block contracts each squeezed patch to a vector of bond dimension $\beta$, these vectors are reassembled into a lower-resolution image, and the process repeats until a final MPS block emits class scores. With $L=4$ layers, input stride 8, inner stride 2, and $\beta=5$, the fixed model reports AUC 0.943 on PCam (versus 0.963 for a rotation-equivariant CNN and 0.962 for a dense CNN baseline), AUC 0.874 on LIDC (versus 0.829 for the dense baseline), and balanced accuracy 0.71 on 3D OASIS (versus 0.67 for the subject-level 3D CNN), while using 0.7 to 2.1 GB rather than 8.8 to 11.5 GB. The paper attributes the memory saving to the absence of large intermediate feature maps and to contraction costs that shrink with depth.
Load-bearing premise
The load-bearing premise is that a decision function that is effectively linear, with no nonlinear activations and, for 3D inputs, no explicit local feature map at all, is expressive enough for these three tasks that the gap to nonlinear CNNs stays small, with batch normalization complicating but not rescuing this assumption.
Editorial extensions
If this is right
- High-resolution 2D and 3D medical images can be classified by tensor networks without flattening them completely, so tensor-network methods are no longer limited to small images.
- The reported memory use of 0.7 to 2.1 GB suggests whole-slide or whole-volume classification could run on ordinary GPUs and allow much larger batches than the 8.8 to 11.5 GB CNN baselines.
- With the architecture fixed and only the bond dimension $\beta$ tuned, a model selected on one dataset transfers to other medical tasks without per-dataset architecture search.
- Because the model is linear, increasing depth mainly reduces computation cost rather than adding representational power, and the practical lever on accuracy is $\beta$, which appears to plateau near $\beta=5$ on LIDC.
- Given the small memory footprint, the main obstacle to wider use shifts to per-epoch computation time, which the paper reports as higher for LoTeNet than for the CNN baselines.
Reading between the lines
- A decisive follow-up the paper does not run is replacing the sinusoidal feature map with a single nonlinear layer; if that closes most of the 0.02 AUC gap on PCam while retaining the memory savings, the linearity assumption will be the binding constraint.
- Comparing LoTeNet against a kernel classifier or linear SVM on the same squeezed patch features would separate what the MPS hierarchy contributes from what the feature lift alone contributes.
- Weight-sharing across MPS blocks, which the paper lists as future work, would make the model translation-equivariant and could reduce its large parameter count; whether that helps histopathology and CT classification is testable.
- The hierarchical contraction pattern is a general low-memory recipe, so analogous constructions could be tried for 3D segmentation or registration, where CNN memory use is often the limiting factor.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript introduces LoTeNet, a tensor-network classifier for 2D and 3D medical images. The method decomposes an image into local patches, each of which is passed through a matrix product state (MPS) block after a 'squeeze' operation that moves spatial information into feature dimensions; the outputs are hierarchically aggregated across several layers to preserve global structure while remaining a linear model in an exponentially high-dimensional feature space. The architecture is fixed across all experiments, with a single tuned hyperparameter (bond dimension beta=5). The authors evaluate on three public datasets: PCam histopathology, LIDC thoracic CT slices, and OASIS MRI volumes, reporting AUC or balanced accuracy and GPU memory usage. They claim LoTeNet achieves performance on par with or superior to CNN baselines while using roughly 0.7-2.1 GB of GPU memory versus 8-11.5 GB for the baselines. The paper also discusses related tensor-network classifiers, computational complexity, and limitations.
Significance. If the central empirical claim is fully supported, the work is significant: it is one of the first tensor-network classifiers to scale to high-resolution 2D and 3D medical images, and its drastically reduced GPU-memory footprint could be valuable in clinical or resource-constrained settings. The exposition is clear, the MPS background and contraction scheme are standard and correctly described, and the authors provide open-source code. The fixed architecture with a single hyperparameter is a strength, as is the honest treatment of limitations in Section 6.5. However, the headline claim of 'on par or superior' performance currently rests on comparisons that are not fully matched and on effect sizes that are not statistically quantified. The significance of the paper will depend on whether these comparisons can be made fair and properly uncertainty-aware.
major comments (4)
- [§5.2.2, Table 1] The PCam comparison to Rotation Eq-CNN uses an AUC of 0.963 taken from Veeling et al. (2018), which was evaluated on the original PCam test set, whereas LoTeNet and the other models in the table are evaluated on the Kaggle re-split after duplicate removal. This is a non-matched comparison: the observed gap between 0.943 (LoTeNet) and 0.963 (Rotation Eq-CNN) may be due at least in part to the different test distributions and preprocessing. The 'comparable' claim in the text is therefore not yet supported. The authors should either re-run Rotation Eq-CNN and Densenet on the same Kaggle split, or clearly state the mismatch and temper the comparison accordingly.
- [§5.2.3, Table 2] On OASIS, the claimed 'clear improvement' of LoTeNet (0.71±0.09) over the subject-level CNN (0.67±0.08) is based on 5-fold cross-validation, and the two intervals overlap substantially. Without a paired statistical test across folds, multiple-seed estimates, or confidence intervals that account for fold correlation, the manuscript does not demonstrate that LoTeNet is superior. The term 'clear improvement' should be removed or a proper significance test (e.g., paired test over the five folds or across repeated runs) must be reported.
- [§5.2.2, Table 1] The PCam and LIDC results are reported as single point estimates with no error bars, no number of runs, and no statistical significance testing. For example, the LIDC AUC of 0.874 for LoTeNet versus 0.829 for Densenet is a single number; without variance over seeds or resamples, the reader cannot assess whether the difference is meaningful. The authors should provide mean and standard deviation over at least a few random seeds, or some equivalent uncertainty quantification, for all entries in Table 1.
- [§5.2.3, §6.3, §6.4] For the 3D OASIS experiments, the sinusoidal local feature map of Eq. (2) is dropped and the model relies only on the squeeze operation; as the manuscript states, LoTeNet optimizes linear decision boundaries with no nonlinear activation functions. Consequently the 3D model is, up to the affine batch normalization at inference, a linear function of the raw voxel intensities. This is an acknowledged property of the method, not an internal inconsistency, but it makes the claimed superiority over nonlinear CNN baselines more surprising and heightens the need for matched statistical evidence. The authors should explicitly discuss whether the linear hypothesis class is sufficiently expressive for these tasks, and ideally include an ablation with a nonlinear local feature map for OASIS to bound the capacity concern.
minor comments (5)
- [§2.1 heading] The heading 'T ensor Network Notations' contains a stray space; it should read 'Tensor Network Notations'.
- [§1, Introduction] The phrase 'magnetric resonance imaging' should be 'magnetic resonance imaging'.
- [§5.2.2] The word 'utlisation' is a typo and should be 'utilisation' or 'utilization'.
- [§3.3] The sentence about binary classification states that output dimension M=1 can be used with the sigmoid non-linearity. Since Eq. (4) uses an argmax over m=0,...,M-1, M=1 is ambiguous; please clarify whether M=2 is used with a single logit or whether the argmax formulation is adjusted.
- [§6.1] The computational complexity expression O(N/k^2L · L·k^2·d·β^2) is typographically ambiguous. Parentheses or an explicit definition of the constants would help the reader verify the claimed exponential reduction.
Circularity Check
No circularity: LoTeNet's construction and evaluation are self-contained; the tuned hyperparameter is ordinary model selection and self-citations are peripheral.
full rationale
LoTeNet is specified by explicit formulas: the squeeze operation in Eq. (7), the MPS factorization in Eq. (6), and the linear decision rule in Eqs. (4)-(5), with end-to-end training via the cross-entropy loss in Eq. (8). None of these definitions are stated in terms of the empirical results they are used to produce. The single hyperparameter beta=5 is selected on the PCam validation set and then fixed for LIDC and OASIS, which is ordinary model selection rather than fitted-input-called-prediction. Performance is measured against external datasets and independently implemented baselines. The paper's self-citations (Selvan and Dam 2020; Selvan et al. 2020, 2021) are used for provenance, for a peripheral remark about computational cost sharing MPS blocks, and for future-work pointers; the central claim does not reduce to these citations. The limitations noted in Section 6.5 (higher wall-clock time, over-parameterization, overfitting tendency) are acknowledged experimental limitations, not circular steps. Concerns about statistical significance or non-matched baseline splits are comparison-validity issues, not equation-level circularity. No load-bearing step reduces by construction to its own input.
Assumptions & free parameters
free parameters (4)
- Bond dimension beta =
5
- Number of layers L =
4
- Input-layer squeeze stride k1 =
8
- Subsequent squeeze stride k_l =
2
assumptions (4)
- domain assumption MPS factorization with beta=5 can approximate the weight tensor well enough for these medical classification tasks
- domain assumption Small image patches can be treated as locally orderless without losing task-relevant structure
- domain assumption Linear decision boundaries in the high-dimensional feature space are sufficient for the medical imaging tasks
- standard math The feature map in Eq. (2) (cos, sin) is a valid unit-norm embedding
Cite this review
Pith. "Pith review of Locally orderless tensor networks for classifying two- and three-dimensional medical images." pith.science (2026). https://pith.science/paper/57PYGEB2
@misc{pith2026200912280,
author = {Pith},
title = {Pith review of: Locally orderless tensor networks for classifying two- and three-dimensional medical images},
year = {2026},
howpublished = {\url{https://pith.science/paper/57PYGEB2}},
note = {Machine review of arXiv:2009.12280}
}
read the original abstract
Tensor networks are factorisations of high rank tensors into networks of lower rank tensors and have primarily been used to analyse quantum many-body problems. Tensor networks have seen a recent surge of interest in relation to supervised learning tasks with a focus on image classification. In this work, we improve upon the matrix product state (MPS) tensor networks that can operate on one-dimensional vectors to be useful for working with 2D and 3D medical images. We treat small image regions as orderless, squeeze their spatial information into feature dimensions and then perform MPS operations on these locally orderless regions. These local representations are then aggregated in a hierarchical manner to retain global structure. The proposed locally orderless tensor network (LoTeNet) is compared with relevant methods on three datasets. The architecture of LoTeNet is fixed in all experiments and we show it requires lesser computational resources to attain performance on par or superior to the compared methods.
Figures
Figures from the paper (6 more)
Reference graph
Works this paper leans on
-
[8]
Alexander Novikov, Mikhail Trofimov, and Ivan Oseledets. Exponential machines.arXiv preprint arXiv:1605.03795,
-
[11]
vdnn: Virtualized deep neural networks for scalable, memory-efficient neural network design
Minsoo Rhu, Natalia Gimelshein, Jason Clemons, Arslan Zulfiqar, and Stephen W Keckler. vdnn: Virtualized deep neural networks for scalable, memory-efficient neural network design. In 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pages 1–13. IEEE,
work page 2016
-
[12]
Segmenting two-dimensional struc- tures with strided tensor networks
Raghavendra Selvan, Erik B Dam, and Jens Petersen. Segmenting two-dimensional struc- tures with strided tensor networks. In 27th International Conference on Information Processing in Medical Imaging (IPMI-2021), Bornholm, Denmark.,
work page 2021
-
[1971]
Matrix Product State Representations
David Perez-Garcia, Frank Verstraete, Michael M Wolf, and J Ignacio Cirac. Matrix product state representations. arXiv preprint quant-ph/0608197,
-
[1992]
The ITensor Software Library for Tensor Network Calculations
Matthew Fishman, Steven R White, and E Miles Stoudenmire. The itensor software library for tensor network calculations.arXiv preprint arXiv:2007.14822,
work page Pith review arXiv 2007
-
[1999]
TensorNetwork for Machine Learning
Stavros Efthymiou, Jack Hidary, and Stefan Leichenauer. Tensornetwork for Machine Learn- ing. arXiv preprint arXiv:1906.06329,
work page Pith review arXiv 1906
-
[2007]
From density-matrix renormalization group to matrix product states
Ian P McCulloch. From density-matrix renormalization group to matrix product states. Journal of Statistical Mechanics: Theory and Experiment, 2007(10):P10014,
work page 2007
-
[2011]
A Multi-Scale Tensor Network Architecture for Classification and Regression
Justin Reyes and Miles Stoudenmire. A multi-scale tensor network architecture for classifi- cation and regression.arXiv preprint arXiv:2001.08286,
work page Pith review arXiv 2001
Show all 13 references
-
[2015]
Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980,
Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization.arXiv preprint arXiv:1412.6980,
-
[2017]
Batchnormalization: Acceleratingdeepnetworktraining by reducing internal covariate shift.arXiv preprint arXiv:1502.03167,
SergeyIoffeandChristianSzegedy. Batchnormalization: Acceleratingdeepnetworktraining by reducing internal covariate shift.arXiv preprint arXiv:1502.03167,
-
[2018]
Renormalization algorithms for quantum-many body systems in two and higher dimensions.arXiv preprint cond-mat/0407066,
Frank Verstraete and J Ignacio Cirac. Renormalization algorithms for quantum-many body systems in two and higher dimensions.arXiv preprint cond-mat/0407066,
-
[2019]
cudnn: Efficient primitives for deep learning
Sharan Chetlur, Cliff Woolley, Philippe Vandermersch, Jonathan Cohen, John Tran, Bryan Catanzaro, and Evan Shelhamer. cudnn: Efficient primitives for deep learning. arXiv preprint arXiv:1410.0759,
-
[2020]
arXiv:2007.03051. Samuel G Armato III, Geoffrey McLennan, Michael F McNitt-Gray, Charles R Meyer, David Yankelevitz, Denise R Aberle, Claudia I Henschke, Eric A Hoffman, Ella A Kazerooni, Heber MacMahon, et al. Lung image database consortium: developing a resource for the medica...
2007 arXiv
Reviewed August 27, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.