REVIEW 4 major objections 4 minor 1 cited by
Generalized Least Squares Kernelized Tensor Factorization
T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read GLSKF claims that adding a locally correlated residual to a smoothed low-rank factorization completes multidimensional tensors more accurately than low-rank baselines.
desk verdict GLSKF has a real algorithmic core and honest attribution, but the empirical superiority claim is currently supported only by oracle-tuned hyperparameters and uneven baselines, so the paper needs a re-run before the headline result can be trusted. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the covariance norm, $\|x\|_{K_x} = \sqrt{x^\top K_x^{-1} x}$, extended to matrices and tensors through Kronecker products of per-mode covariance matrices; this is what turns smoothness priors into quadratic regularizers. Around it the model adds two structures: the CP decomposition $M = \sum_{r=1}^R u_1^r \circ \cdots \circ u_D^r$ for the global component, and a residual tensor $R$ regularized by a sparse, tapered product covariance $K_R$. Updating the latent factors requires solving $\left(H_d^\top O_d'^\top O_d' H_d + \rho (K_d^u)^{-1} \otimes I_R\right) \operatorname{vec}(U_d^\top) = b$, where the binary matrices $O_d$ and $O_d'$ slice and zero-pad missing entries without destroying the Kronecker structure, so conjugate gradient can apply the system one matrix–vector product at a time. The residual update uses the dual form $r = K_R O_1^\top (O_1 K_R O_1^\top + \gamma I_{|\Omega|})^{-1} l_o$, which reduces the solve to the observed subset size $|\Omega|$ and keeps each CG product cheap through Kronecker and sparse multiplications.
What would settle it
Re-run the four experiments with the tuning parameters fixed by cross-validation on the observed entries only, and compare GLSKF against LSKF, SPC, GRALS, TRMF, and HaLRTC under identical held-out masks; if GLSKF no longer ranks first across the board, the empirical superiority claim is not established.
Extended reading notes
Core claim
On the paper's own terms, GLSKF establishes that a complementary additive decomposition — a covariance-regularized low-rank global tensor plus a locally correlated residual tensor under a generalized least squares objective — can jointly model global dependencies and localized variations in multidimensional data completion. The paper introduces a covariance norm that measures smoothness through inverse covariance matrices and treats smoothness constraints such as quadratic variation, graph Laplacians, and autoregressive regularization as special cases. It then develops an alternating least squares solver in which every update reduces to a linear system solved by conjugate gradient, with projection matrices that preserve the Kronecker product structure of the per-mode covariances even under missing data. The reported experiments show GLSKF outperforming HaLRTC, SPC, GRALS, TRMF, and its own ablations on traffic imputation, image inpainting, video recovery, and MRI reconstruction.
Load-bearing premise
The reported performance depends on the choice of several tuning parameters, and the experiments pick those parameters using the ground-truth entries that the method is then asked to predict.
Editorial extensions
If this is right
- GLSKF completes tensors with a per-iteration cost that is linear in the number of observed entries and the mode sizes, because every linear system is applied through Kronecker and sparse matrix–vector products rather than formed and inverted.
- Because the local component absorbs high-frequency detail, the global CP rank can stay small (R = 10–20 in the experiments), which cuts cost while preserving accuracy.
- The covariance norm unifies existing smoothness regularizers, so GLSKF extends quadratic variation, graph Laplacian, autoregressive, and Gaussian-process smoothness constraints within one objective.
- The same formulation applies to third-order images and fourth-order video, with per-mode kernels including identity covariances for channels and learned covariances for the local residual.
- Across the four tasks, GLSKF reports better reconstruction than the compared low-rank baselines at missing rates from 30% to 99%.
Reading between the lines
- Because the experiments select $\rho$, $\gamma$, kernel length-scales, and tapering ranges using the missing ground truth, the reported margins should be rechecked with parameters chosen from observed entries alone; the advantage may shrink.
- The additive structure can be read as a low-rank-plus-locally-correlated-noise model, which suggests using the estimated residual component R for detecting abrupt events, such as traffic incidents, that should concentrate there.
- For regular-grid images and video, replacing tapered-kernel multiplications with FFT-based Toeplitz multiplications should push per-iteration cost toward $O(R I_d \log I_d)$, a direction the paper leaves open.
- A direct extension is denoising and corrupted-data recovery with only a small change to the objective, as the paper notes; one could test GLSKF against plug-and-play deep priors on those tasks.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes GLSKF, a generalized least squares kernelized tensor factorization framework for completing multidimensional tensor data. The model additively combines a low-rank CP decomposition with covariance-norm smoothness constraints on the latent factors and a separate locally correlated residual component regularized by sparse/tapered covariance matrices. Estimation is done by alternating least squares with conjugate-gradient solves that exploit Kronecker and sparsity structure, giving a claimed per-iteration cost that is linear in the number of observed entries. The authors evaluate the method on traffic speed imputation, color image inpainting, color video completion, and MRI reconstruction, reporting superior reconstruction metrics and favorable runtime compared with HaLRTC, SPC, GRALS, TRMF, and ablation variants LSTF, LSKF, and GLSlocal.
Significance. If the empirical claims were validated, GLSKF would be a useful contribution: it provides a principled additive decomposition of global smooth structure and local correlation, with a deterministic and apparently scalable optimization scheme, and the authors release the code. The Kronecker-structured CG derivation in Section IV is internally coherent and the complexity analysis in Section VI is plausible. However, the central empirical claim—that GLSKF achieves superior reconstruction performance across all four tasks—is currently not supported because the tuning parameters are selected using the test ground truth, and the baselines are not given an equivalent tuning budget. This is not a fatal flaw in the method itself but requires a reworked evaluation before the paper can be accepted.
major comments (4)
- [§IV.C, §V.A.2, §V.B.2, §V.C.2, §V.D.2] The manuscript states in Section IV.C that 'these parameters are tuned based on available ground truth' and repeats in each experimental section that ρ and γ are selected 'based on the ground truth.' In a tensor-completion experiment, the ground truth is precisely the set of missing entries being predicted; choosing ρ, γ, length-scales, and tapering ranges from these values makes every reported MAE/RMSE/PSNR an in-sample evaluation rather than an estimate of out-of-sample performance. This is the central empirical claim of the paper, so the evaluation must be rerun with a validation-based selection protocol (e.g., a held-out subset of observed entries, cross-validation, or model selection using only observed data) or with a sensitivity analysis showing that the reported margins persist over a wide range of hyperparameters.
- [§V.A.2, §V.B.2, §V.C.2, §V.D.2] The comparison does not give baselines the same tuning budget: HaLRTC's regularization parameter is selected from only {10^{-4}, 10^{-5}} in Sections V.A.2 and V.B.2 and fixed to 10^{-2} in Section V.D.2, SPC's smoothness parameters are either fixed or reported as 'best' without describing the search, and no repeated runs or variance measures are reported for any method. Since the headline result is the margin over these baselines, the experiments should report a common tuning protocol (same search grids, same validation or cross-validation criterion, same number of trials) and include standard deviations or confidence intervals over multiple missing-data masks.
- [§VI.B, Fig. 8] The wall-clock comparison in Figure 8 is not apples-to-apples: the caption states that for Carphone SPC is stopped at 2,000 iterations 'as it shows slow convergence,' while GLSKF is run with its own convergence criterion. To support the scalability claim, all methods should be run under the same convergence criterion or compared at matched time budgets, and the iteration counts and stopping tolerance for each method should be reported.
- [§IV.C, §V.A.2] The kernel length-scales and tapering ranges are 'selected empirically' and described as 'interpretable and align well with the characteristics of the test datasets,' but no procedure is given for how these values are chosen or how sensitive the results are to them. Because these hyperparameters are part of the oracle tuning described in the previous comments, the paper needs at least a sensitivity analysis or a clearly defined selection rule to demonstrate that the reported margins are not artifacts of hand-picking the kernel scales.
minor comments (4)
- [§V.A.2] The paragraph beginning 'For baselines with structures similar to GLSKF (LSTF, LSKF, GLSlocal)...' is repeated verbatim in Section V.A.2; the duplicate should be removed.
- [§IV.A] The definitions of O_d and O'_d are easy to confuse; please state explicitly that O'_d is the row-selector for the vectorized mode-d unfolding and relate it to O_d with a displayed equation.
- [Table I] In Table I, the 'Closed-form' row cites Eq. (20) for both the |Ω|^3 cost and the GLSKF CG cost; clarify which equation and which variant is being referenced in each column.
- [Fig. 8] The MSE curves in Figure 8 would be easier to read on a log scale, since the early GLSKF trajectory dominates the vertical range for House256.
Circularity Check
Oracle tuning of GLSKF hyperparameters on the missing ground truth makes reported superiority margins in-sample fits; rerunning with validation-based selection is required before the empirical claim is supported.
-
fitted input called prediction
[Section IV.C (Model Parameters) and Section V.A.2 (Configuration); metrics defined in Section V introduction]
"In practice, these parameters are tuned based on available ground truth. ... we select ρ from {1, 5, 10, 15, 20} and choose γ from {1/10, 1/5, 1, 5, 10} based on the ground truth. ... we compare model performance using mean absolute error (MAE) and root mean square error (RMSE) on the test (missing) data."
The reported errors are computed on the test (missing) entries, and the same ground truth is used to tune ρ and γ (and, per Section IV.C, other hyperparameters). Any selection of ρ/γ that minimizes test error is by construction an in-sample fit to the values being predicted. Thus the claimed margins over LSTF, LSKF, GLSlocal, HaLRTC, SPC, GRALS, and TRMF are not out-of-sample estimates; they reduce to oracle model selection on the target labels. Kernel length-scales and tapering ranges are also selected empirically from test-data characteristics, compounding the leakage. No validation split or independent selection rule is described. The GLSKF objective and CG/ALS derivations are self-contained; the circularity is in the empirical evaluation of the central performance claim.
full rationale
The mathematical derivation chain—covariance norm (Def. 1), objective (Eq. 6), ALS updates (Eqs. 9, 14, 20), and Kronecker/CG complexity analysis—is self-contained and does not reduce to its inputs. Citations to the authors' prior BCKL work [17] are used for provenance of the additive structure, not as load-bearing proof, so no self-citation circularity is found. However, the empirical claim that GLSKF is superior across traffic, image, video, and MRI completion is compromised: Section IV.C states that ρ and γ are tuned based on available ground truth, and Section V.A.2 repeats that ρ and γ are selected based on the ground truth, while the reported metrics are evaluated on the test (missing) data. Because the missing entries are the prediction targets, using them to select regularization parameters turns the reported errors into fitted values. Asymmetric baseline tuning (e.g., HaLRTC's regularization parameter restricted to two values, SPC's best result reported) further weakens the comparison, though that asymmetry is a fairness concern rather than circularity itself. The central mathematical contribution can still stand, but the headline empirical margin requires validation-based hyperparameter selection before it is supported.
Assumptions & free parameters
free parameters (6)
- CP rank R =
R=20 for traffic; R=10 for image/video/MRI
- Regularization weight rho =
chosen from {1,5,10,15,20} per scenario
- Regularization weight gamma =
chosen from {0.1,0.2,1,5,10} per scenario
- Kernel length-scales for latent covariances K_u_d =
e.g., l=1 (traffic spatial), l=40 (traffic time), l=30 (image/video pixels), l=5 (video frames)
- Kernel length-scales and tapering ranges for residual covariances K_r_d =
e.g., l=5 with lambda=10 or 30 depending on dataset and dimension
- Empirical residual covariance K_r_D for the channel/depth dimension =
updated each iteration as cov(R_(D))
assumptions (4)
- standard math Kronecker product identities and conjugate gradient convergence for positive definite systems
- domain assumption The true data is well approximated by the sum of a low-rank CP tensor and a locally correlated residual with separable Kronecker covariance
- domain assumption Missing entries are missing completely at random in the experiments
- ad hoc to paper The chosen kernels (regularized Laplacian, Matérn, Bohman taper) are appropriate for each modality
invented entities (1)
-
Locally correlated residual tensor R
Cite this review
Pith. "Pith review of Generalized Least Squares Kernelized Tensor Factorization." pith.science (2026). https://pith.science/paper/25RD2CF5
@misc{pith2026241207041,
author = {Pith},
title = {Pith review of: Generalized Least Squares Kernelized Tensor Factorization},
year = {2026},
howpublished = {\url{https://pith.science/paper/25RD2CF5}},
note = {Machine review of arXiv:2412.07041}
}
read the original abstract
Recovering incomplete multidimensional tensor-structured data is a fundamental task in many real-world applications. Smoothness-constrained low-rank tensor factorization effectively captures global and long-range correlations, but often struggles to characterize short-scale, high-frequency, or locally varying structures. We propose GLSKF, a complementary Generalized Least Squares Kernelized Tensor Factorization framework, for multidimensional spatiotemporal data completion. GLSKF additively integrates a covariance-regularized low-rank global component with an explicitly modeled locally correlated residual component under a GLS objective, enabling effective modeling of both global dependencies and localized variations. A covariance norm regularizer encodes spatiotemporal dependencies in both components: structured covariances are imposed on the latent factor columns to enforce smoothness in the global factorization, whereas compactly supported sparse kernels are used to model local correlations in the residual. We develop an alternating least squares algorithm with blockwise linear-system updates that exploit the Kronecker structure of the covariance matrices under missing data and facilitate fast conjugate gradient solves. Additional computational gains are obtained by exploiting the sparsity and Toeplitz structure of the local residual covariance matrices for efficient matrix-vector multiplications. We evaluate GLSKF on four real-world multidimensional data-completion tasks: traffic speed imputation, color image completion, digital video recovery, and MRI data reconstruction. Experimental results demonstrate that GLSKF achieves superior reconstruction performance and favorable scalability across a range of tensor completion tasks, supporting its broad applicability to multidimensional data completion.
Figures
Figures from the paper (5 more)
Forward citations
Cited by 1 Pith paper
-
Robust Tensor Completion via Gradient Tensor Nulclear L1-L2 Norm for Traffic Data Recovery
A gradient-domain tensor L1-L2 norm regularizer jointly encodes low rank and local smoothness, enabling robust completion of traffic tensors corrupted by missing entries and noise.
Reference graph
Works this paper leans on
-
[17]
Bayesian Complementary Kernelized Learning for Multidimensional Spatiotemporal Data
M. Lei, A. Labbe, and L. Sun, “Bayesian complementary kernel- ized learning for multidimensional spatiotemporal data,” arXiv preprint arXiv:2208.09978, 2022
work page Pith review arXiv 2022
-
[20]
A generalized least-square matrix decomposition,
G. I. Allen, L. Grosenick, and J. Taylor, “A generalized least-square matrix decomposition,” Journal of the American Statistical Association , vol. 109, no. 505, pp. 145–159, 2014
work page 2014
-
[21]
Tensor de- composition meets rkhs: Efficient algorithms for smooth and misaligned data,
B. W. Larsen, T. G. Kolda, A. R. Zhang, and A. H. Williams, “Tensor de- composition meets rkhs: Efficient algorithms for smooth and misaligned data,” arXiv preprint arXiv:2408.05677 , 2024
arXiv 2024
-
[1]
Low-rank autoregressive tensor completion for spatiotemporal traffic data imputation,
X. Chen, M. Lei, N. Saunier, and L. Sun, “Low-rank autoregressive tensor completion for spatiotemporal traffic data imputation,” IEEE Transactions on Intelligent Transportation Systems , vol. 23, no. 8, pp. 12 301–12 310, 2021
work page 2021
-
[2]
Bayesian kernelized matrix factorization for spatiotemporal traffic data imputation and kriging,
M. Lei, A. Labbe, Y . Wu, and L. Sun, “Bayesian kernelized matrix factorization for spatiotemporal traffic data imputation and kriging,” IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 10, pp. 18 962–18 974, 2022
2022
-
[3]
H. Chen, M. Lin, J. Liu, H. Yang, C. Zhang, and Z. Xu, “Nt-dptc: a non-negative temporal dimension preserved tensor completion model for missing traffic data imputation,” Information Sciences , vol. 653, p. 119797, 2024
work page 2024
-
[4]
Tensor methods in computer vision and deep learning,
Y . Panagakis, J. Kossaifi, G. G. Chrysos, J. Oldfield, M. A. Nicolaou, A. Anandkumar, and S. Zafeiriou, “Tensor methods in computer vision and deep learning,” Proceedings of the IEEE , vol. 109, no. 5, pp. 863– 890, 2021
work page 2021
-
[5]
A. Cichocki, D. Mandic, L. De Lathauwer, G. Zhou, Q. Zhao, C. Caiafa, and H. A. Phan, “Tensor decompositions for signal processing applica- tions: From two-way to multiway component analysis,” IEEE signal processing magazine, vol. 32, no. 2, pp. 145–163, 2015
work page 2015
Show all 33 references
-
[6]
Tensor decompositions for hyperspectral data processing in remote sensing: A comprehensive review,
M. Wang, D. Hong, Z. Han, J. Li, J. Yao, L. Gao, B. Zhang, and J. Chanussot, “Tensor decompositions for hyperspectral data processing in remote sensing: A comprehensive review,” IEEE Geoscience and Remote Sensing Magazine , vol. 11, no. 1, pp. 26–72, 2023
2023
-
[7]
Scalable tensor factorizations for incomplete data,
E. Acar, D. M. Dunlavy, T. G. Kolda, and M. Mørup, “Scalable tensor factorizations for incomplete data,” Chemometrics and Intelligent Laboratory Systems, vol. 106, no. 1, pp. 41–56, 2011
2011
-
[8]
Tensor completion for estimating missing values in visual data,
J. Liu, P. Musialski, P. Wonka, and J. Ye, “Tensor completion for estimating missing values in visual data,” IEEE transactions on pattern analysis and machine intelligence , vol. 35, no. 1, pp. 208–220, 2012
2012
-
[9]
Tensor ring decompo- sition with rank minimization on latent space: An efficient approach for tensor completion,
L. Yuan, C. Li, D. Mandic, J. Cao, and Q. Zhao, “Tensor ring decompo- sition with rank minimization on latent space: An efficient approach for tensor completion,” in Proceedings of the AAAI conference on artificial intelligence, vol. 33, no. 01, 2019, pp. 9151–9158
2019
-
[10]
Smooth parafac decomposition for tensor completion,
T. Yokota, Q. Zhao, and A. Cichocki, “Smooth parafac decomposition for tensor completion,” IEEE Transactions on Signal Processing, vol. 64, no. 20, pp. 5423–5436, 2016
2016
-
[11]
Collaborative filtering with graph information: Consistency and scalable methods,
N. Rao, H.-F. Yu, P. Ravikumar, and I. S. Dhillon, “Collaborative filtering with graph information: Consistency and scalable methods,” Advances in Neural Information Processing Systems , pp. 2107–2115, 2015
2015
-
[12]
Fast multivariate spatio-temporal analysis via low rank tensor learning,
M. T. Bahadori, Q. R. Yu, and Y . Liu, “Fast multivariate spatio-temporal analysis via low rank tensor learning,” Advances in Neural Information Processing Systems, pp. 3491–3499, 2014
2014
-
[13]
Learning from multiway data: Simple and efficient tensor regression,
R. Yu and Y . Liu, “Learning from multiway data: Simple and efficient tensor regression,” International Conference on Machine Learning , pp. 373–381, 2016
2016
-
[14]
Traffic data recon- struction via adaptive spatial-temporal correlations,
Y . Wang, Y . Zhang, X. Piao, H. Liu, and K. Zhang, “Traffic data recon- struction via adaptive spatial-temporal correlations,” IEEE Transactions on Intelligent Transportation Systems , vol. 20, no. 4, pp. 1531–1543, 2018
2018
-
[15]
Temporal regularized matrix factorization for high-dimensional time series prediction,
H.-F. Yu, N. Rao, and I. S. Dhillon, “Temporal regularized matrix factorization for high-dimensional time series prediction,” Advances in neural information processing systems , vol. 29, 2016. 15
2016
-
[16]
Temporal collaborative filtering with bayesian probabilistic tensor fac- torization,
L. Xiong, X. Chen, T.-K. Huang, J. Schneider, and J. G. Carbonell, “Temporal collaborative filtering with bayesian probabilistic tensor fac- torization,” in Proceedings of the 2010 SIAM international conference on data mining . SIAM, 2010, pp. 211–222
2010
-
[18]
Kernelized probabilistic matrix factorization: Exploiting graphs and side information,
T. Zhou, H. Shan, A. Banerjee, and G. Sapiro, “Kernelized probabilistic matrix factorization: Exploiting graphs and side information,” in Pro- ceedings of the 2012 SIAM international Conference on Data mining . SIAM, 2012, pp. 403–414
2012
-
[19]
Scalable spatiotemporally varying coef- ficient modeling with bayesian kernelized tensor regression,
M. Lei, A. Labbe, and L. Sun, “Scalable spatiotemporally varying coef- ficient modeling with bayesian kernelized tensor regression,” Bayesian Analysis, vol. 1, no. 1, pp. 1–29, 2024
2024
-
[22]
KI Williams and C
C. KI Williams and C. E. Rasmussen, Gaussian processes for machine learning. MIT press Cambridge, MA, 2006
2006
-
[23]
A full scale approximation of covariance functions for large spatial data sets,
H. Sang and J. Z. Huang, “A full scale approximation of covariance functions for large spatial data sets,” Journal of the Royal Statistical Society: Series B (Statistical Methodology) , vol. 74, no. 1, pp. 111–132, 2012
2012
-
[24]
Bayesian nonstationary spatial modeling for very large datasets,
M. Katzfuss, “Bayesian nonstationary spatial modeling for very large datasets,” Environmetrics, vol. 24, no. 3, pp. 189–200, 2013
2013
-
[25]
Functional data analysis by matrix completion,
M.-H. Descary and V . M. Panaretos, “Functional data analysis by matrix completion,” The Annals of Statistics , vol. 47, no. 1, pp. 1–38, 2019
2019
-
[26]
Random surface covariance estimation by shifted partial tracing,
T. Masak and V . M. Panaretos, “Random surface covariance estimation by shifted partial tracing,” Journal of the American Statistical Associa- tion, pp. 1–13, 2022
2022
-
[27]
Tensor completion via complementary global, local, and nonlocal priors,
X.-L. Zhao, J.-H. Yang, T.-H. Ma, T.-X. Jiang, M. K. Ng, and T.- Z. Huang, “Tensor completion via complementary global, local, and nonlocal priors,” IEEE Transactions on Image Processing , vol. 31, pp. 984–999, 2021
2021
-
[28]
Tensor decompositions and applications,
T. G. Kolda and B. W. Bader, “Tensor decompositions and applications,” SIAM Review, vol. 51, no. 3, pp. 455–500, 2009
2009
-
[29]
Covariance tapering for interpolation of large spatial datasets,
R. Furrer, M. G. Genton, and D. Nychka, “Covariance tapering for interpolation of large spatial datasets,” Journal of Computational and Graphical Statistics, vol. 15, no. 3, pp. 502–523, 2006
2006
-
[30]
Covariance tapering for likelihood-based estimation in large spatial data sets,
C. G. Kaufman, M. J. Schervish, and D. W. Nychka, “Covariance tapering for likelihood-based estimation in large spatial data sets,” Journal of the American Statistical Association , vol. 103, no. 484, pp. 1545–1555, 2008
2008
-
[31]
Rue and L
H. Rue and L. Held, Gaussian Markov random fields: theory and applications. Chapman and Hall/CRC, 2005
2005
-
[32]
Thoughts on massively scalable gaussian processes,
A. G. Wilson, C. Dann, and H. Nickisch, “Thoughts on massively scalable gaussian processes,” arXiv preprint arXiv:1511.01870 , 2015
2015 arXiv
-
[33]
Compactly supported correlation functions,
T. Gneiting, “Compactly supported correlation functions,” Journal of Multivariate Analysis, vol. 83, no. 2, pp. 493–508, 2002. Mengying Lei received the M.S. degree from the School of Automation Science and Electrical Engineering, Beihang University, Beijing, China, in 2019, a...
2002
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.