Pith. sign in

REVIEW 4 major objections 5 minor 19 references

Tensor-Train Parameterization for Ultra Dimensionality Reduction

T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Pith's one-line read The paper proposes TTPUDR, which tensorizes the locality-preserving projection as a tensor-train and swaps the squared-error objective for a Frobenius-norm one, yielding a low-parameter projection that preserves spatial structure and…

desk verdict Genuinely new TT-plus-LPP combination with real storage-efficiency potential, but the robustness claim is undercut by a false l1/F-norm motivation, unproven convergence, and weak empirical support. read the letter →

arxiv 1908.04924 v1 pith:ASTSJLFS submitted 2019-08-14 cs.LG eess.IVstat.ML

classification cs.LGeess.IVstat.ML
keywords tensor-traindecompositionlocalitypreservingprojectionsdimensionalityreductionFrobeniusnormrobustnesstooutliersStiefelmanifoldhigh-dimensionaltensordatagraphembedding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes a dimensionality reduction method, TTPUDR, that turns the classical locality preserving projections (LPP) mapping into a chain of small tensor cores—a tensor-train—instead of a single large matrix, and measures projection error with the Frobenius norm rather than the squared error. The goal is to make LPP work on ultra-high-dimensional tensor data without first flattening it to a vector, while using far fewer parameters and resisting extreme outliers. The paper reports that on face and hyperspectral image data TTPUDR beats PCA and LPP in classification accuracy when the data dimension exceeds the sample count, and that on data with added block noise it keeps accuracy much higher than the baselines. The load-bearing consequence is that high-order tensor structure can be exploited for dimensionality reduction without paying exponential storage or computation costs.

What carries the argument

The central object is the tensor-train (TT) parameterization: an $n$-mode tensor is written as a contraction of $n$ three-way cores $U_1,\dots,U_n$, which costs $O(nIR^2)$ storage instead of the full tensor size. The argument is carried by replacing the LPP objective's squared Frobenius distance with the plain Frobenius norm and then rewriting it as a squared norm with data-dependent weights $\tilde{s}_{ij}$; this turns each core update into a small orthogonality-constrained (Stiefel manifold) eigenproblem built from matrices $H_k$, and the affinity weights are refreshed after every sweep. The orthogonality constraints $L^{\top}(U_k)L(U_k)=I_{R_k}$ guarantee the overall projection has orthogonal columns.

What would settle it

Run TTPUDR from many random tensor-core initializations on the noisy Yale B setting and compare the final objective values and classification accuracies; if different initializations settle at materially different values or the objective does not stabilize, the alternating reweighted scheme is not solving the claimed optimization problem.

Watch

Extended reading notes

Core claim

On its own terms, the paper establishes that the LPP projection, normally a matrix solving a generalized eigenvalue problem, can be replaced by a tensor-train parameterized mapping $\tilde{U}=U_1 \times U_2 \times \cdots \times U_n$ whose left unfolding $L(\tilde{U})$ projects a tensor input by contraction. The squared $F$-norm in the standard LPP objective is replaced by an unsquared Frobenius norm, and the optimization is recast as a sequence of smaller subproblems over orthogonality-constrained tensor cores, with the similarity graph reweighted by $\tilde{s}_{ij} = s_{ij}/\|L^{\top}(\tilde{U})V(X_i)-L^{\top}(\tilde{U})V(X_j)\|_F$ after each sweep. The paper shows this yields classification models with $O(nIR^2)$ effective parameters, operating where the data dimension is larger than the number of samples, and reports accuracy gains over PCA and LPP on the Extended Yale B face data and Indian Pines hyperspectral data, including under 10% and 20% block noise.

Load-bearing premise

The method assumes the alternating sweep—recompute the local weights from the current projection, then optimize each tensor core—converges to a minimizer of the stated Frobenius-norm objective, a convergence the paper explicitly says has not been proven.

Editorial extensions

If this is right

  • Storage cost of the learned projection scales as $O(nIR^2)$, with $n$ modes, maximum mode size $I$, and tensor rank $R$, so the parameter count stays far below the $O(p)$ matrices of PCA and LPP.
  • The method can run when the data dimension exceeds the training sample size, as demonstrated on 1024-dimensional face patches with 482 training samples, a regime where LPP cannot execute.
  • On the noisy Extended Yale B data, TTPUDR keeps overall accuracy well above PCA and LPP at both 10% and 20% block noise, and reaches the higher accuracy at a lower reduced dimensionality.
  • Each tensor-core subproblem is a small Stiefel-manifold optimization, so the algorithm avoids forming the full vectorized data matrix that would be infeasible for ultra-high-dimensional data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the $\tilde{s}_{ij}$ reweighting is structurally an iteratively reweighted least-squares scheme, so a proof of monotone decrease or fixed-point convergence would turn TTPUDR into a template for robustifying other graph-embedding objectives.
  • Editorial inference: because the mapping is a chain of mode-wise cores, the same construction should extend naturally to streaming or time-series tensors, although the paper only lists that as future work.
  • Editorial inference: the experiments report classification accuracy only; a direct measure of local geometric preservation, such as nearest-neighbor reconstruction error, would test more sharply whether the tensor-train subspace really preserves spatial relations as claimed.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes TTPUDR, a dimensionality reduction method that parameterizes an LPP-style projection as a tensor-train and replaces the squared-Frobenius LPP objective with a non-squared Frobenius-norm objective (Problem (5)), claimed to be a robust approximation to the ℓ1-norm. The optimization alternates between reweighting the affinity matrix via Equation (6) and solving weighted squared-Frobenius subproblems (Problems (10)–(12)) on Stiefel manifolds. Experiments on Extended Yale B and Indian Pines compare TTPUDR with PCA and LPP, reporting better accuracy on face data and better robustness in a small-sample noisy setting.

Significance. The idea of using a tensor-train parameterization to compress the projection operator is potentially useful, and the reported storage complexity O(nIR^2) is a concrete advantage over the Tucker-based TLPP. The empirical gain on Extended Yale B is large and, if reproducible, would be of interest. However, the paper's central theoretical motivation is mathematically incorrect (the Frobenius norm is not an approximation of the ℓ1-norm), the convergence of the proposed alternating scheme is explicitly unproved, and the experiments do not cover the claimed ultra-high-dimensional regime or the stated state-of-the-art baselines. No code or data are released, and the evaluation protocol contains internal inconsistencies. As it stands, the support for the central claims is insufficient.

major comments (4)
  1. [Section II.A] The statement that "[a]n approximation to the ℓ1-norm is the Frobenius norm" is false: by the paper's own definition, ‖X‖_F = ‖V(X)‖_2, which is an ℓ2-norm, not an ℓ1-norm. This false premise is load-bearing because Contribution 2 and the robustness claims attribute the method's outlier resistance to this alleged ℓ1 approximation. The non-squared Frobenius objective in Problem (5) is a legitimate robust alternative to the squared Frobenius norm, but it is not an ℓ1-type objective, and the paper should either establish the robustness property by a correct argument or substantially weaken the claim.
  2. [Section III, Algorithm 1] The convergence of the alternating reweighted scheme is unproved and, as written, not guaranteed. The weights ~s_ij in Equation (6) depend on the current tensor cores, and fixing them in Problem (7) does not produce a global majorization of Problem (5), so the original objective need not decrease monotonically. A zero or near-zero denominator in Equation (6) can also produce unbounded weights. The paper itself concedes that "there has not been any perfect theoretical proof of the convergence of TTPUDR." Without a proof of convergence to a stationary point of Problem (5), or at least a rigorous empirical convergence analysis with multiple initializations, the reported classification gains cannot be attributed to solving the stated objective rather than to initialization or early stopping.
  3. [Section IV] The experimental evaluation does not support the abstract's claim that TTPUDR "significantly outperforms the past methods and the several state-of-the-art methods." Only PCA and LPP are compared; TTNPE and TLPP are dismissed without running. On the Indian Pines dataset (Table I), TTPUDR is substantially worse than both PCA and LPP (OA 0.7101 versus 0.7907 and 0.7810). Moreover, the first two experimental settings have data dimension smaller than the number of samples (200 < 10,366 and 1024 < 2414), so they do not test the ultra-high-dimensional regime that is the paper's stated motivation; only the 20%-training noisy case (482 samples vs. 1024 dimensions) violates LPP's assumption, and in that case the LPP baseline is absent. The evidence for the central performance claims is therefore incomplete and partly contradictory.
  4. [Section III, Eqs. (5)–(12)] The output dimension R_n is defined inconsistently. In the text preceding Problem (5), R_n is set to I_1 I_2 ... I_n, which would mean the projection has the same dimensionality as the vectorized input and no reduction occurs. Algorithm 1 line 1 and the experiments instead use R_n = 2, ..., 30 as the number of features. Since the subproblems (10)–(12) depend critically on the meaning of R_n (e.g., the slices Y_k(:, :, r_n, :, :) and the constraint L^⊤(U_n)L(U_n) = I_{R_n}), this inconsistency affects the mathematical formulation of the method and must be resolved.
minor comments (5)
  1. [Section IV.A] The data-split description for the noised dataset is contradictory: "60% and 20% of the data are treated to be the training set and 40% and 80% of the data are set as the test set" does not define coherent experimental settings and should be rewritten.
  2. [Section IV.B] The evaluation metrics are nonstandard: the average accuracy formula uses TP/(TP+FP) rather than per-class recall, and the Kappa coefficient formula is not the usual Cohen's kappa. Since all methods are evaluated with the same formulas, this may not change relative rankings, but it should be corrected or justified.
  3. [Section IV.C.1] The results in Tables I and II are reported for a single randomly selected output dimension (R_n = 24 and R_n = 28), while the text says R_n ranges from 2 to 30. Reporting only one dimension makes the comparison fragile and does not show the behavior across the claimed range; the authors should present results across all R_n or justify the selection.
  4. [Section II.A] The claim that the Frobenius norm "preserves the general spatial relations between elements" while the ℓ1-norm "treats each element separately and can probably cause the spatial information loss" is not substantiated. Both norms are elementwise functions of the tensor entries; neither explicitly exploits spatial layout. If the intention is that the projection operator itself preserves tensor structure, that point should be argued separately.
  5. [Section I] The statement that "the tensor-train decomposition is proved to be able to efficiently and effectively capture the spatial relations" is too strong without a specific reference or argument. The cited work [7] is about tensor train subspace and neighborhood preserving embedding, not a general proof of spatial-relation capture.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: TTPUDR's derivation is self-contained; the unproven convergence of the reweighted scheme is a correctness risk, not a circularity.

full rationale

The paper's central derivation replaces the F-norm LPP objective (5) with an iteratively reweighted squared-Frobenius objective (7), where the weights in Eq. (6) are defined from the current projected distances. This is the standard IRLS construction: at a fixed point, the first-order stationarity condition of the weighted squared objective coincides with that of the original F-norm objective, so the reduction is an optimization equivalence, not a presupposition of the conclusion. No parameter is fitted to class labels or to the evaluation metric; the projection cores are learned from the unlabeled graph/affinity S, and classification accuracy is measured afterward with a 1NN classifier on an independent test split. The paper cites prior work on tensor-train decomposition and LPP for standard definitions, and the only specialized cited lemma (orthogonality of the TT map, Lemma 2 in [7]) is used to justify a constraint, not to import the paper's own claim. The admitted absence of a convergence proof for Algorithm 1 ('there has not been any perfect theoretical proof of the convergence of TTPUDR') is a genuine limitation and a correctness/reproducibility concern, but it is not circular reasoning: an algorithm that fails to converge is not deriving its conclusion from its premises. The experimental comparisons are against external baselines (PCA, LPP) on public datasets; no benchmark is used to define the model. Therefore no step reduces by construction to its input, and the circularity score is 0.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical or mathematical entities. Its free parameters are the tensor ranks and graph hyperparameters, all hand-chosen. The load-bearing assumptions are the representational power of TT for the mapping, the cited orthogonality lemma, and, critically, an unproved convergence of the alternating reweighted optimization.

free parameters (4)
  • TT ranks R_1..R_{n-1} = e.g., 1x4x3, 3x5x4, 4x10xR_n for Indiana; 1x4x4, 4x8x7, 7x4x4, 4x8xR_n for Yale B
    Chosen by hand per dataset; control the capacity and storage complexity of the TT mapping and are not derived from data or an objective criterion.
  • Output dimension R_n = 2 to 30; selected values 24 (Indiana) and 28 (Yale B) reported
    The reduced dimensionality is a user choice; the reported values are presented inconsistently as random and best.
  • Heat kernel width t = 0.02 (Indiana), 0.5 (Yale B)
    Set manually for the affinity graph; directly shapes the locality weights in LPP and TTPUDR.
  • Number of neighbors k = 4 for both datasets
    Chosen by hand for k-NN graph construction in the affinity matrix.
assumptions (4)
  • domain assumption TT decomposition represents the mapping tensor compactly with O(nIR^2) storage and sufficient expressive power
    Relies on Oseledets' TT results (cited [10]); the mapping is assumed to be well approximated by this low-rank tensor network.
  • domain assumption Orthogonality of each core's left unfolding L(Uk) implies orthogonality of the overall mapping columns
    Invoked via Lemma 2 in [7] (Remark 1); accepted without proof in this paper.
  • ad hoc to paper The alternating reweighted scheme of Equations (6)-(7) converges to a minimizer of the F-norm objective (5)
    The authors state 'there has not been any perfect theoretical proof of the convergence of TTPUDR' in Section III; the algorithm's validity depends on this fixed-point iteration.
  • domain assumption The k-NN heat-kernel graph captures local structure relevant for classification
    Standard assumption from LPP [12]; errors in the graph directly corrupt the projection.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Tensor-Train Parameterization for Ultra Dimensionality Reduction." pith.science (2026). https://pith.science/paper/ASTSJLFS

@misc{pith2026190804924,
  author       = {Pith},
  title        = {Pith review of: Tensor-Train Parameterization for Ultra Dimensionality Reduction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ASTSJLFS}},
  note         = {Machine review of arXiv:1908.04924}
}
read the original abstract

Locality preserving projections (LPP) are a classical dimensionality reduction method based on data graph information. However, LPP is still responsive to extreme outliers. LPP aiming for vectorial data may undermine data structural information when it is applied to multidimensional data. Besides, it assumes the dimension of data to be smaller than the number of instances, which is not suitable for high-dimensional data. For high-dimensional data analysis, the tensor-train decomposition is proved to be able to efficiently and effectively capture the spatial relations. Thus, we propose a tensor-train parameterization for ultra dimensionality reduction (TTPUDR) in which the traditional LPP mapping is tensorized in terms of tensor-trains and the LPP objective is replaced with the Frobenius norm to increase the robustness of the model. The manifold optimization technique is utilized to solve the new model. The performance of TTPUDR is assessed on classification problems and TTPUDR significantly outperforms the past methods and the several state-of-the-art methods.

Figures

Figures reproduced from arXiv: 1908.04924 by the authors.

Figure 1
Figure 1. Comparison of overall accuracy (OA) for TTPUDR, LPP a [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 18 canonical work pages

  1. [1]

    Multilinear analysis of image en- sembles: Tensorfaces,

    M. V asilescu and D. Terzopoulos, “Multilinear analysis of image en- sembles: Tensorfaces,” in ECCV, A. Heyden, G. Sparr, M. Nielsen, and P . Johansen, Eds., 2002, pp. 447–460

  2. [2]

    Matrix and tensor decomposition in reco mmender systems,

    P . Symeonidis, “Matrix and tensor decomposition in reco mmender systems,” in ACM RecSys, 2016, pp. 429–430

  3. [3]

    Tensor Decompositions for Signal Processing Applications From Two-way to Multiway Component Analysis

    A. Cichocki, D. Mandic, A. Phan, C. Caiafa, G. Zhou, Q. Zha o, and L. Lathauwer, “Tensor decompositions for signal processin g applications from two-way to multiway component analysis,” arXiv:1403.4462, 2014

  4. [4]

    Tensorial extensions of i ndepen- dent component analysis for multisubject FMRI analysis,

    C. F. Beckmann and S. M. Smith, “Tensorial extensions of i ndepen- dent component analysis for multisubject FMRI analysis,” Neuroimage, vol. 25, no. 1, pp. 294–311, 2005

  5. [5]

    Tensor completion by alternating minimization under the tensor train (TT) model,

    W. Wang, V . Aggarwal, and S. Aeron, “Tensor completion by alternating minimization under the tensor train (TT) model,” arXiv;1609.05587, 2016

  6. [6]

    Tensor embedding methods,

    G. Dai and D. Y eung, “Tensor embedding methods,” in AAAI, 2006, pp. 330–335

  7. [7]

    Principal Component Analysis with Tensor Train Subspace

    W. Wang, V . Aggarwal, and S. Aeron, “Principal component analysis with tensor train subspace,” arXiv:1803.05026, 2018

  8. [8]

    Multiple invariants and generalized r ank of a p-way matrix or tensor,

    F. L. Hitchcock, “Multiple invariants and generalized r ank of a p-way matrix or tensor,” Journal of Mathematics and Physics , vol. 7, no. 1-4, pp. 39–79, 1928

Show all 19 references
  1. [9]

    Implications of factor analysis of three- way matrices for measurement of change,

    L. R. Tucker, “Implications of factor analysis of three- way matrices for measurement of change,” in Problems in measuring change , C. W. Harris, Ed. Madison WI: U of Wisconsin Press, 1963, pp. 122–1 37

  2. [10]

    Tensor-train decomposition,

    I. Oseledets, “Tensor-train decomposition,” SIAM Journal on Scientific Computing, vol. 33, no. 5, pp. 2295–2317, 2011

  3. [11]

    Liii. on lines and planes of closest fit to sy stems of points in space,

    K. Pearson, “Liii. on lines and planes of closest fit to sy stems of points in space,” The London, Edinburgh, and Dublin Philosophical Magazine and Journal of Science , vol. 2, no. 11, pp. 559–572, 1901

  4. [12]

    Locality preserving projections,

    X. He and P . Niyogi, “Locality preserving projections, ” in Advances in Neural Information Processing Systems , S. Thrun, L. Saul, and B. Sch¨ olkopf, Eds., vol. 16. Cambridge, MA: MIT Press, 2004

  5. [13]

    Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions,

    A. Cichocki, N. Lee, I. Oseledets, A.-H. Phan, Q. Zhao, a nd D. P . Mandic, “Tensor networks for dimensionality reduction and large-scale optimization: Part 1 low-rank tensor decompositions,” F oundations and Trends in Machine Learning , vol. 9, no. 4-5, pp. 249–429, 2016

  6. [14]

    Cichocki, N

    A. Cichocki, N. Lee, A. Phan, I. Oseledets, Q. Zhao, and D. Mandic, Tensor Networks for Dimensionality Reduction and Large-Scale Optimization: Part 2 Applications and Futu re Perspectives, ser. Foundations and Trends(r) in Machine Learning Series. Now Publishers, 2017. [Online...

  7. [15]

    Breaking the curse o f dimensionality, or how to use SVD in many dimensions,

    I. Oseledets and E. Tyrtyshnikov, “Breaking the curse o f dimensionality, or how to use SVD in many dimensions,” SIAM Journal on Scientific Computing, vol. 31, no. 5, pp. 3744–3759, 2009

  8. [16]

    Lpp solution schemes for use with face recognition,

    Y . Xu, A. Zhong, J. Y ang, and D. Zhang, “Lpp solution schemes for use with face recognition,” Pattern Recognition , vol. 43, no. 12, pp. 4165–4176, Dec. 2010. [Online]. Availab le: http://dx.doi.org/10.1016/j.patcog.2010.06.016

  9. [17]

    Tensor train neighb orhood pre- serving embedding,

    W. Wang, V . Aggarwal, and S. Aeron, “Tensor train neighb orhood pre- serving embedding,” IEEE Transactions on Signal Processing , vol. 66, no. 10, pp. 2724–2732, May 2018

  10. [18]

    From fe w to many: Illumination cone models for face recognition under variab le lighting and pose,

    A. Georghiades, P . Belhumeur, and D. Kriegman, “From fe w to many: Illumination cone models for face recognition under variab le lighting and pose,” IEEE Transactions on Pattern Analysis and Machine Intellig ence, vol. 23, no. 6, pp. 643–660, 2001

  11. [19]

    220 band aviris hyperspectral image data set: June 12, 1992 indian pine test site 3,

    M. F. Baumgardner, L. L. Biehl, and D. A. Landgrebe, “220 band aviris hyperspectral image data set: June 12, 1992 indian pine test site 3,” Sep 2015

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.