REVIEW 4 major objections 5 minor 19 references
Tensor-Train Parameterization for Ultra Dimensionality Reduction
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read The paper proposes TTPUDR, which tensorizes the locality-preserving projection as a tensor-train and swaps the squared-error objective for a Frobenius-norm one, yielding a low-parameter projection that preserves spatial structure and…
desk verdict Genuinely new TT-plus-LPP combination with real storage-efficiency potential, but the robustness claim is undercut by a false l1/F-norm motivation, unproven convergence, and weak empirical support. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the tensor-train (TT) parameterization: an $n$-mode tensor is written as a contraction of $n$ three-way cores $U_1,\dots,U_n$, which costs $O(nIR^2)$ storage instead of the full tensor size. The argument is carried by replacing the LPP objective's squared Frobenius distance with the plain Frobenius norm and then rewriting it as a squared norm with data-dependent weights $\tilde{s}_{ij}$; this turns each core update into a small orthogonality-constrained (Stiefel manifold) eigenproblem built from matrices $H_k$, and the affinity weights are refreshed after every sweep. The orthogonality constraints $L^{\top}(U_k)L(U_k)=I_{R_k}$ guarantee the overall projection has orthogonal columns.
What would settle it
Run TTPUDR from many random tensor-core initializations on the noisy Yale B setting and compare the final objective values and classification accuracies; if different initializations settle at materially different values or the objective does not stabilize, the alternating reweighted scheme is not solving the claimed optimization problem.
Extended reading notes
Core claim
On its own terms, the paper establishes that the LPP projection, normally a matrix solving a generalized eigenvalue problem, can be replaced by a tensor-train parameterized mapping $\tilde{U}=U_1 \times U_2 \times \cdots \times U_n$ whose left unfolding $L(\tilde{U})$ projects a tensor input by contraction. The squared $F$-norm in the standard LPP objective is replaced by an unsquared Frobenius norm, and the optimization is recast as a sequence of smaller subproblems over orthogonality-constrained tensor cores, with the similarity graph reweighted by $\tilde{s}_{ij} = s_{ij}/\|L^{\top}(\tilde{U})V(X_i)-L^{\top}(\tilde{U})V(X_j)\|_F$ after each sweep. The paper shows this yields classification models with $O(nIR^2)$ effective parameters, operating where the data dimension is larger than the number of samples, and reports accuracy gains over PCA and LPP on the Extended Yale B face data and Indian Pines hyperspectral data, including under 10% and 20% block noise.
Load-bearing premise
The method assumes the alternating sweep—recompute the local weights from the current projection, then optimize each tensor core—converges to a minimizer of the stated Frobenius-norm objective, a convergence the paper explicitly says has not been proven.
Editorial extensions
If this is right
- Storage cost of the learned projection scales as $O(nIR^2)$, with $n$ modes, maximum mode size $I$, and tensor rank $R$, so the parameter count stays far below the $O(p)$ matrices of PCA and LPP.
- The method can run when the data dimension exceeds the training sample size, as demonstrated on 1024-dimensional face patches with 482 training samples, a regime where LPP cannot execute.
- On the noisy Extended Yale B data, TTPUDR keeps overall accuracy well above PCA and LPP at both 10% and 20% block noise, and reaches the higher accuracy at a lower reduced dimensionality.
- Each tensor-core subproblem is a small Stiefel-manifold optimization, so the algorithm avoids forming the full vectorized data matrix that would be infeasible for ultra-high-dimensional data.
Reading between the lines
- Editorial inference: the $\tilde{s}_{ij}$ reweighting is structurally an iteratively reweighted least-squares scheme, so a proof of monotone decrease or fixed-point convergence would turn TTPUDR into a template for robustifying other graph-embedding objectives.
- Editorial inference: because the mapping is a chain of mode-wise cores, the same construction should extend naturally to streaming or time-series tensors, although the paper only lists that as future work.
- Editorial inference: the experiments report classification accuracy only; a direct measure of local geometric preservation, such as nearest-neighbor reconstruction error, would test more sharply whether the tensor-train subspace really preserves spatial relations as claimed.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes TTPUDR, a dimensionality reduction method that parameterizes an LPP-style projection as a tensor-train and replaces the squared-Frobenius LPP objective with a non-squared Frobenius-norm objective (Problem (5)), claimed to be a robust approximation to the ℓ1-norm. The optimization alternates between reweighting the affinity matrix via Equation (6) and solving weighted squared-Frobenius subproblems (Problems (10)–(12)) on Stiefel manifolds. Experiments on Extended Yale B and Indian Pines compare TTPUDR with PCA and LPP, reporting better accuracy on face data and better robustness in a small-sample noisy setting.
Significance. The idea of using a tensor-train parameterization to compress the projection operator is potentially useful, and the reported storage complexity O(nIR^2) is a concrete advantage over the Tucker-based TLPP. The empirical gain on Extended Yale B is large and, if reproducible, would be of interest. However, the paper's central theoretical motivation is mathematically incorrect (the Frobenius norm is not an approximation of the ℓ1-norm), the convergence of the proposed alternating scheme is explicitly unproved, and the experiments do not cover the claimed ultra-high-dimensional regime or the stated state-of-the-art baselines. No code or data are released, and the evaluation protocol contains internal inconsistencies. As it stands, the support for the central claims is insufficient.
major comments (4)
- [Section II.A] The statement that "[a]n approximation to the ℓ1-norm is the Frobenius norm" is false: by the paper's own definition, ‖X‖_F = ‖V(X)‖_2, which is an ℓ2-norm, not an ℓ1-norm. This false premise is load-bearing because Contribution 2 and the robustness claims attribute the method's outlier resistance to this alleged ℓ1 approximation. The non-squared Frobenius objective in Problem (5) is a legitimate robust alternative to the squared Frobenius norm, but it is not an ℓ1-type objective, and the paper should either establish the robustness property by a correct argument or substantially weaken the claim.
- [Section III, Algorithm 1] The convergence of the alternating reweighted scheme is unproved and, as written, not guaranteed. The weights ~s_ij in Equation (6) depend on the current tensor cores, and fixing them in Problem (7) does not produce a global majorization of Problem (5), so the original objective need not decrease monotonically. A zero or near-zero denominator in Equation (6) can also produce unbounded weights. The paper itself concedes that "there has not been any perfect theoretical proof of the convergence of TTPUDR." Without a proof of convergence to a stationary point of Problem (5), or at least a rigorous empirical convergence analysis with multiple initializations, the reported classification gains cannot be attributed to solving the stated objective rather than to initialization or early stopping.
- [Section IV] The experimental evaluation does not support the abstract's claim that TTPUDR "significantly outperforms the past methods and the several state-of-the-art methods." Only PCA and LPP are compared; TTNPE and TLPP are dismissed without running. On the Indian Pines dataset (Table I), TTPUDR is substantially worse than both PCA and LPP (OA 0.7101 versus 0.7907 and 0.7810). Moreover, the first two experimental settings have data dimension smaller than the number of samples (200 < 10,366 and 1024 < 2414), so they do not test the ultra-high-dimensional regime that is the paper's stated motivation; only the 20%-training noisy case (482 samples vs. 1024 dimensions) violates LPP's assumption, and in that case the LPP baseline is absent. The evidence for the central performance claims is therefore incomplete and partly contradictory.
- [Section III, Eqs. (5)–(12)] The output dimension R_n is defined inconsistently. In the text preceding Problem (5), R_n is set to I_1 I_2 ... I_n, which would mean the projection has the same dimensionality as the vectorized input and no reduction occurs. Algorithm 1 line 1 and the experiments instead use R_n = 2, ..., 30 as the number of features. Since the subproblems (10)–(12) depend critically on the meaning of R_n (e.g., the slices Y_k(:, :, r_n, :, :) and the constraint L^⊤(U_n)L(U_n) = I_{R_n}), this inconsistency affects the mathematical formulation of the method and must be resolved.
minor comments (5)
- [Section IV.A] The data-split description for the noised dataset is contradictory: "60% and 20% of the data are treated to be the training set and 40% and 80% of the data are set as the test set" does not define coherent experimental settings and should be rewritten.
- [Section IV.B] The evaluation metrics are nonstandard: the average accuracy formula uses TP/(TP+FP) rather than per-class recall, and the Kappa coefficient formula is not the usual Cohen's kappa. Since all methods are evaluated with the same formulas, this may not change relative rankings, but it should be corrected or justified.
- [Section IV.C.1] The results in Tables I and II are reported for a single randomly selected output dimension (R_n = 24 and R_n = 28), while the text says R_n ranges from 2 to 30. Reporting only one dimension makes the comparison fragile and does not show the behavior across the claimed range; the authors should present results across all R_n or justify the selection.
- [Section II.A] The claim that the Frobenius norm "preserves the general spatial relations between elements" while the ℓ1-norm "treats each element separately and can probably cause the spatial information loss" is not substantiated. Both norms are elementwise functions of the tensor entries; neither explicitly exploits spatial layout. If the intention is that the projection operator itself preserves tensor structure, that point should be argued separately.
- [Section I] The statement that "the tensor-train decomposition is proved to be able to efficiently and effectively capture the spatial relations" is too strong without a specific reference or argument. The cited work [7] is about tensor train subspace and neighborhood preserving embedding, not a general proof of spatial-relation capture.
Circularity Check
No circularity: TTPUDR's derivation is self-contained; the unproven convergence of the reweighted scheme is a correctness risk, not a circularity.
full rationale
The paper's central derivation replaces the F-norm LPP objective (5) with an iteratively reweighted squared-Frobenius objective (7), where the weights in Eq. (6) are defined from the current projected distances. This is the standard IRLS construction: at a fixed point, the first-order stationarity condition of the weighted squared objective coincides with that of the original F-norm objective, so the reduction is an optimization equivalence, not a presupposition of the conclusion. No parameter is fitted to class labels or to the evaluation metric; the projection cores are learned from the unlabeled graph/affinity S, and classification accuracy is measured afterward with a 1NN classifier on an independent test split. The paper cites prior work on tensor-train decomposition and LPP for standard definitions, and the only specialized cited lemma (orthogonality of the TT map, Lemma 2 in [7]) is used to justify a constraint, not to import the paper's own claim. The admitted absence of a convergence proof for Algorithm 1 ('there has not been any perfect theoretical proof of the convergence of TTPUDR') is a genuine limitation and a correctness/reproducibility concern, but it is not circular reasoning: an algorithm that fails to converge is not deriving its conclusion from its premises. The experimental comparisons are against external baselines (PCA, LPP) on public datasets; no benchmark is used to define the model. Therefore no step reduces by construction to its input, and the circularity score is 0.
Assumptions & free parameters
free parameters (4)
- TT ranks R_1..R_{n-1} =
e.g., 1x4x3, 3x5x4, 4x10xR_n for Indiana; 1x4x4, 4x8x7, 7x4x4, 4x8xR_n for Yale B
- Output dimension R_n =
2 to 30; selected values 24 (Indiana) and 28 (Yale B) reported
- Heat kernel width t =
0.02 (Indiana), 0.5 (Yale B)
- Number of neighbors k =
4 for both datasets
assumptions (4)
- domain assumption TT decomposition represents the mapping tensor compactly with O(nIR^2) storage and sufficient expressive power
- domain assumption Orthogonality of each core's left unfolding L(Uk) implies orthogonality of the overall mapping columns
- ad hoc to paper The alternating reweighted scheme of Equations (6)-(7) converges to a minimizer of the F-norm objective (5)
- domain assumption The k-NN heat-kernel graph captures local structure relevant for classification
Cite this review
Pith. "Pith review of Tensor-Train Parameterization for Ultra Dimensionality Reduction." pith.science (2026). https://pith.science/paper/ASTSJLFS
@misc{pith2026190804924,
author = {Pith},
title = {Pith review of: Tensor-Train Parameterization for Ultra Dimensionality Reduction},
year = {2026},
howpublished = {\url{https://pith.science/paper/ASTSJLFS}},
note = {Machine review of arXiv:1908.04924}
}
read the original abstract
Locality preserving projections (LPP) are a classical dimensionality reduction method based on data graph information. However, LPP is still responsive to extreme outliers. LPP aiming for vectorial data may undermine data structural information when it is applied to multidimensional data. Besides, it assumes the dimension of data to be smaller than the number of instances, which is not suitable for high-dimensional data. For high-dimensional data analysis, the tensor-train decomposition is proved to be able to efficiently and effectively capture the spatial relations. Thus, we propose a tensor-train parameterization for ultra dimensionality reduction (TTPUDR) in which the traditional LPP mapping is tensorized in terms of tensor-trains and the LPP objective is replaced with the Frobenius norm to increase the robustness of the model. The manifold optimization technique is utilized to solve the new model. The performance of TTPUDR is assessed on classification problems and TTPUDR significantly outperforms the past methods and the several state-of-the-art methods.
Figures
Reference graph
Works this paper leans on
-
[1]
Multilinear analysis of image en- sembles: Tensorfaces,
M. V asilescu and D. Terzopoulos, “Multilinear analysis of image en- sembles: Tensorfaces,” in ECCV, A. Heyden, G. Sparr, M. Nielsen, and P . Johansen, Eds., 2002, pp. 447–460
work page 2002
-
[2]
Matrix and tensor decomposition in reco mmender systems,
P . Symeonidis, “Matrix and tensor decomposition in reco mmender systems,” in ACM RecSys, 2016, pp. 429–430
work page 2016
-
[3]
Tensor Decompositions for Signal Processing Applications From Two-way to Multiway Component Analysis
A. Cichocki, D. Mandic, A. Phan, C. Caiafa, G. Zhou, Q. Zha o, and L. Lathauwer, “Tensor decompositions for signal processin g applications from two-way to multiway component analysis,” arXiv:1403.4462, 2014
work page Pith review arXiv 2014
-
[4]
Tensorial extensions of i ndepen- dent component analysis for multisubject FMRI analysis,
C. F. Beckmann and S. M. Smith, “Tensorial extensions of i ndepen- dent component analysis for multisubject FMRI analysis,” Neuroimage, vol. 25, no. 1, pp. 294–311, 2005
work page 2005
-
[5]
Tensor completion by alternating minimization under the tensor train (TT) model,
W. Wang, V . Aggarwal, and S. Aeron, “Tensor completion by alternating minimization under the tensor train (TT) model,” arXiv;1609.05587, 2016
arXiv 2016
-
[6]
G. Dai and D. Y eung, “Tensor embedding methods,” in AAAI, 2006, pp. 330–335
work page 2006
-
[7]
Principal Component Analysis with Tensor Train Subspace
W. Wang, V . Aggarwal, and S. Aeron, “Principal component analysis with tensor train subspace,” arXiv:1803.05026, 2018
work page Pith review arXiv 2018
-
[8]
Multiple invariants and generalized r ank of a p-way matrix or tensor,
F. L. Hitchcock, “Multiple invariants and generalized r ank of a p-way matrix or tensor,” Journal of Mathematics and Physics , vol. 7, no. 1-4, pp. 39–79, 1928
work page 1928
Show all 19 references
-
[9]
Implications of factor analysis of three- way matrices for measurement of change,
L. R. Tucker, “Implications of factor analysis of three- way matrices for measurement of change,” in Problems in measuring change , C. W. Harris, Ed. Madison WI: U of Wisconsin Press, 1963, pp. 122–1 37
1963
-
[10]
Tensor-train decomposition,
I. Oseledets, “Tensor-train decomposition,” SIAM Journal on Scientific Computing, vol. 33, no. 5, pp. 2295–2317, 2011
2011
-
[11]
Liii. on lines and planes of closest fit to sy stems of points in space,
K. Pearson, “Liii. on lines and planes of closest fit to sy stems of points in space,” The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science , vol. 2, no. 11, pp. 559–572, 1901
1901
-
[12]
Locality preserving projections,
X. He and P . Niyogi, “Locality preserving projections, ” in Advances in Neural Information Processing Systems , S. Thrun, L. Saul, and B. Sch¨ olkopf, Eds., vol. 16. Cambridge, MA: MIT Press, 2004
2004
-
[13]
Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions,
A. Cichocki, N. Lee, I. Oseledets, A.-H. Phan, Q. Zhao, a nd D. P . Mandic, “Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions,” F oundations and Trends in Machine Learning , vol. 9, no. 4-5, pp. 249–429, 2016
2016
-
[14]
Cichocki, N
A. Cichocki, N. Lee, A. Phan, I. Oseledets, Q. Zhao, and D. Mandic, Tensor Networks for Dimensionality Reduction and Large-Scale Optimization: Part 2 Applications and Futu re Perspectives, ser. Foundations and Trends(r) in Machine Learning Series. Now Publishers, 2017. [Online...
2017
-
[15]
Breaking the curse o f dimensionality, or how to use SVD in many dimensions,
I. Oseledets and E. Tyrtyshnikov, “Breaking the curse o f dimensionality, or how to use SVD in many dimensions,” SIAM Journal on Scientific Computing, vol. 31, no. 5, pp. 3744–3759, 2009
2009
-
[16]
Lpp solution schemes for use with face recognition,
Y . Xu, A. Zhong, J. Y ang, and D. Zhang, “Lpp solution schemes for use with face recognition,” Pattern Recognition , vol. 43, no. 12, pp. 4165–4176, Dec. 2010. [Online]. Availab le: http://dx.doi.org/10.1016/j.patcog.2010.06.016
2010 doi
-
[17]
Tensor train neighb orhood pre- serving embedding,
W. Wang, V . Aggarwal, and S. Aeron, “Tensor train neighb orhood pre- serving embedding,” IEEE Transactions on Signal Processing , vol. 66, no. 10, pp. 2724–2732, May 2018
2018
-
[18]
From fe w to many: Illumination cone models for face recognition under variab le lighting and pose,
A. Georghiades, P . Belhumeur, and D. Kriegman, “From fe w to many: Illumination cone models for face recognition under variab le lighting and pose,” IEEE Transactions on Pattern Analysis and Machine Intellig ence, vol. 23, no. 6, pp. 643–660, 2001
2001
-
[19]
220 band aviris hyperspectral image data set: June 12, 1992 indian pine test site 3,
M. F. Baumgardner, L. L. Biehl, and D. A. Landgrebe, “220 band aviris hyperspectral image data set: June 12, 1992 indian pine test site 3,” Sep 2015
1992
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.