Stochastic Gradient Descent for Incomplete Tensor Linear Systems

Alexander Xue; Anna Ma; Deanna Needell

REVIEW 2 major objections 4 minor 24 references

This paper adapts stochastic gradient descent to solve tensor linear systems when data are missing under uniform, column-block, or frontal-slice models, and proves convergence rates.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 10:56 UTC pith:MMJVVPUA

load-bearing objection Useful general-C framework for tensor SGD with missing data, but the constant-step convergence proof rests on a false co-coercivity lemma and the experiments don't match the stated assumptions. the 2 major comments →

arxiv 2510.07630 v2 pith:MMJVVPUA submitted 2025-10-08 math.NA cs.NA

Stochastic Gradient Descent for Incomplete Tensor Linear Systems

Anna Ma , Deanna Needell , Alexander Xue This is my paper

classification math.NA cs.NA MSC 65F1015A6965K10

keywords tensor recoveryt-productmissing datastochastic gradient descentincomplete tensor linear systemscolumn block missingfrontal slice missingconvergence

verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper extends a stochastic gradient descent method for tensor linear systems from the previously studied uniform missing-data case to two other missingness models and one that exists only for tensors. Its key step is modifying the SGD update direction with a 0/1 correction tensor so that the random gradient estimate remains unbiased even though entries of the data tensor are missing. Under mild assumptions, the method converges in expectation with O((2+log t)/√t) objective error for decaying step sizes and, for fixed step sizes, achieves a linear contraction rate down to a convergence horizon. The authors exhibit the correct correction tensor for uniform, column-block, and frontal-slice missingness and confirm the behavior numerically. A sympathetic reader would care because missing tensor data are common in video and sensor applications, and this gives one algorithm and one analysis that adapt to different missingness structures.

Core claim

The central discovery is that the difficulty of missing tensor data can be reduced to choosing a 0/1 tensor C such that the expected value of the masked Gram tensor A~*_{i::} * A~_{i::} decomposes as p C∘(A*_{i::} A_{i::}) plus p² (1-C)∘(...). When this holds for every row, the update direction g(X) = (1/p²)(A~*_{i::}*(A~_{i::} X - p B_{i::})) - ((1-p)/p²) C∘(A~*_{i::} A~_{i::}) X is an unbiased estimator of the gradient ∇F(X) = (1/m) A* (A X - B). The paper shows three concrete models fit this condition: uniform entry missingness with C as the frontal diagonal, column-block missingness with C block-diagonal in every frontal slice, and frontal-slice missingness with C supported only on the z

What carries the argument

The load-bearing object is the correction tensor C, a 0/1 tensor that marks which products in the masked Gram tensor A~*_{i::} * A~_{i::} keep the coefficient p rather than p² when a random row's missing entries are averaged. Equations (3) and (4) are the unbiasedness conditions: E[C∘(A~*_{i::} A~_{i::})] = p C∘(A*_{i::} A_{i::}) and E[(1-C)∘(A~*_{i::} A~_{i::})] = p²(1-C)∘(A*_{i::} A_{i::}). The t-product, defined through block-circulant matrices, lets the whole argument run with tensor notions of transpose, norm, and gradient in close analogy to the matrix case.

Load-bearing premise

Everything for the varying-step result rests on a 0/1 correction tensor C satisfying (3)-(4) for every row; the constant-step result additionally rests on the inequality ∥g(X)-g(Y)∥² ≤ L⟨X-Y,g(X)-g(Y)⟩, which is guaranteed only when g is the gradient of a convex function, and the paper's signed-quadratic construction of the potential f does not ensure convexity.

What would settle it

With n=1, A=[1,-1] as the single data row, p=0.5, B=0, and the uniform-model C (the identity), the potential f for the update g is ½ Xᵀ H X with H = (1/p²)(Aᵀ A - (1-p)I), whose eigenvalues are -2 and 6. For X=0 and Y=(1,1), the co-coercivity inequality reads 8 ≤ -24 and fails. This directly checks the constant-step proof's key inequality for a valid instance of the method; if the theorem is to survive, it needs a route that does not rely on convexity of f.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

For any of the three missingness models, running Algorithm 1 with step size C/√t drives expected objective error to zero at the same O((2+log t)/√t) rate as the uniform case.
A fixed step size α < 1/L_g contracts the expected squared iterate error by factor r = 1 - 2αµ(1-αL_g) each iteration, down to a convergence horizon αG*/(µ(1-αL_g)); a practical schedule is fast fixed-step descent followed by decaying steps.
The frontal-slice missing model has no matrix analogue, so the tensor setting genuinely needs the new analysis rather than a direct translation of matrix results.
The framework is not tied to one noise model: if another missingness pattern admits a 0/1 correction tensor satisfying (3) and (4), the same convergence theorems apply.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The correction-tensor template suggests a recipe for other missingness structures: partition entries into groups that vanish together, and set C to mark pairs in the same group; the same proof should carry through as long as one 0/1 C works simultaneously for every row.
The frontal-slice model highlights that t-product Gram matrices mix all frontal slices; missingness in one slice can contaminate the estimated Gram entries of another slice, so correction must be designed at the level of the block-circulant structure, not entrywise.
A testable practical prediction from the bounds is that the convergence-horizon radius scales roughly as 1/(p³ m) in the row norms for small p, so heavier missingness should visibly slow the fixed-step phase; the paper's synthetic plots appear consistent with this, and a controlled experiment could verify it.
The fixed-step theorem's proof route requires the update map to satisfy a co-coercivity inequality that is guaranteed only for gradients of convex functions; if non-convex potentials arise, either an added condition or an alternative argument is needed for that theorem.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Desk Editor's Note

Useful general-C framework for tensor SGD with missing data, but the constant-step convergence proof rests on a false co-coercivity lemma and the experiments don't match the stated assumptions.

read the letter

Here's my take. The core idea is good and usable: the correction tensor C with conditions (3)-(4) is a clean way to encode what a missing-data model does to the row-wise Gram products, and it genuinely generalizes the uniform model to column-block and frontal-slice missingness. The frontal-slice model has no matrix analogue, and the construction of C for it is neat. Theorem 1 is a routine application of a standard SGD lemma, and the second-moment bound in Lemma 4 is worked out carefully. I believe the diminishing-step convergence claim holds up.

The soft spots are real. Lemma 2 is stated as co-coercivity for any gradient of a smooth function with L-Lipschitz g; that is only true if the potential f is convex. Lemma 8 constructs an f whose Hessian is not PSD in general — you get an indefinite matrix even in the matrix case, e.g. one row [1, -1] and p = 0.5. So step (c) in the proof of Theorem 2 is unjustified. This is load-bearing: the constant-step linear rate to a convergence horizon is the paper's advertised practical result, and the experiments switch step sizes because of it. The theorem might be salvageable by proving a Lipschitz-based bound with a smaller step size or by imposing a PSD condition on the random matrices, but the current proof does not go through.

Smaller issue: Assumption 1 says 'every row chosen at most once', but that does not give the conditional unbiasedness SGD needs. The experiments sample rows without replacement, which violates the analysis. Sampling with replacement or redrawing the mask each iteration would fix the protocol. The code is public, which is good.

In balance: the framework is a real advance in a niche area, the paper is clearly written, and the flaws are specific and fixable. It deserves peer review, with attention to the Theorem 2 proof and a clarified Assumption 1.

Referee Report

2 major / 4 minor

Summary. The paper proposes mSGDT, an SGD-based iterative method for solving tensor linear systems A*X=B under the t-product when the coefficient tensor A is only partially observed. The method uses a modified update direction g(X) with a correction term involving a 0/1 tensor C, designed so that E[g(X)] = ∇F(X) whenever C satisfies conditions (3) and (4). The paper states two convergence results: Theorem 1 gives an O(log t / sqrt(t)) bound under a decaying step size, and Theorem 2 gives a linear contraction rate to a convergence horizon under a constant step size. It constructs explicit C tensors for three missing-data models—uniform, column-block, and frontal-slice—and reports synthetic and video experiments. The framework generalizes earlier work by the same authors from the matrix case and from the uniform-missing tensor case.

Significance. The paper's general C-condition framework is a genuine contribution: it isolates the algebraic condition needed for an unbiased stochastic gradient and applies it to missing-data models beyond the uniform one, including a tensor-specific frontal-slice model. Proposition 1 and the verification of conditions (3) and (4) for the three models are correct. Theorem 1, conditional on Assumptions 1 and 2, follows from a standard SGD lemma, and the explicit G bound in the appendix is a useful concrete estimate. The writing is clear, the proofs are detailed, and the authors provide reproducible code. However, the stronger advertised result, Theorem 2, is not established as written because its proof uses a co-coercivity inequality that requires convexity of a function that Lemma 8 constructs with indefinite Hessian. There is also an unresolved issue with the without-replacement row sampling permitted by Assumption 1. These are load-bearing for the constant-step linear-rate claim, which motivates the two-stage step-size schedule used in the experiments.

major comments (2)

[Section 3, Lemma 2 and proof of Theorem 2, step (c)] Lemma 2 is co-coercivity, which is valid only when f is convex. Lemma 8 constructs f as a signed sum of quadratic terms f(X) = Σ α_j/2 ||u_j^* * X||^2 with α_j ∈ {±1}; such an f is not convex in general, and the construction itself does not guarantee nonnegative α_j. A concrete counterexample occurs already under the uniform missing model: take n = 1, row A_i = [1, -1], p = 0.5, both entries observed, and B = 0. Then g(X) = MX with M = (1/p^2)(A_i^T A_i - (1-p)I) = [[2, -4], [-4, 2]], which has eigenvalues 6 and -2. For v = [1, 1], ||g(v) - g(0)||^2 = 8 while ⟨v, g(v) - g(0)⟩ = -4, so Lemma 2's inequality would require 8 ≤ L(-4), impossible for any L > 0. Therefore the replacement in step (c), which uses Lemma 2 to bound E_{t-1}||g(X_{t-1}) - g(X*)||^2, is unjustified. Since this step produces the contraction factor r = 1 - 2αμ(1 - αL_g), the constant-step linear-rate bound and the conve
[Assumption 1 and proofs of Theorems 1 and 2] The first alternative in Assumption 1—'every row A_i:: is chosen at most once during Algorithm 1'—does not support the unbiasedness property used in the proofs. If rows are sampled without replacement, then conditionally on the previous iterations the next row is uniform over the remaining rows, not over all m rows. Consequently E_{t-1}[g(X_{t-1})] ≠ ∇F(X_{t-1}) in general, which is exactly the equality used in step (a) of the proof of Theorem 2 and is also needed for the application of Lemma 1 in Theorem 1. The second alternative (mask redrawn each iteration) combined with i.i.d. uniform row sampling would suffice, but the without-replacement branch does not. Since the experiments in Section 4.1 explicitly sample rows without replacement, the gap is directly relevant to the reported validation. The authors should either restrict the analysis to i.i.d. row sampling with replacement, or p

minor comments (4)

[Throughout] The symbol C is used both for the 0/1 correction tensor and for the step-size constant in Theorem 1 (α_t = C/√t). This is confusing, especially since Theorem 2 uses L_g with a related notation. Rename the step-size constant, e.g., η.
[Section 1.2] Typo: 'the the first two models' should be 'the first two models'.
[Proof of Theorem 2, step (b)] There is a missing parenthesis in the displayed bound: 'E_{t-1}[||g(X_{t-1} - g(X*)||^2]' should read 'E_{t-1}[||g(X_{t-1}) - g(X*)||^2]'.
[Algorithm 1 and Assumption 1] The sampling mechanism should be stated precisely. Algorithm 1 says 'Choose row index i ∈ [m] uniformly at random,' which conventionally means with replacement, but Assumption 1 and the experiments allow without replacement. Clarify which sampling model is used and how the proofs apply to it.

Circularity Check

0 steps flagged

No significant circularity: conditions (3)-(4) are verified per model and convergence follows from standard SGD lemmas; self-citations are context, not load-bearing.

full rationale

The key reduction is the unbiased-gradient identity (5): given Assumptions 1-2 and a 0/1 tensor C satisfying (3)-(4), E[g(X)] = ∇F(X) by direct expectation computation. This is an ex ante verification, not a fitted prediction: C is not chosen to match the convergence bound, and the bound follows from the standard SGD lemma [20]. Proposition 1 independently constructs C for the uniform, column-block, and frontal-slice missing-data models by direct computation of E_D[(a~_j^* a~_k)_{xy}], yielding (6)-(7) and hence (3)-(4). The new models are not defined in terms of the target theorems, and the uniform-model case is re-proved in Proposition 1. The paper cites the same authors' prior works [1] and [11] as context and proof templates (e.g., 'These lemmas closely follow the proof steps done in [1]' and 'Lemma 7 is a restatement of a lemma appearing in [1]'), but the central convergence claims do not reduce to those citations. The possible failure of Lemma 2's co-coercivity when the auxiliary f in Lemma 8 is nonconvex is a correctness issue, not circularity, since it does not smuggle the desired conclusion into the inputs. Accordingly, the circularity burden is very low.

Axiom & Free-Parameter Ledger

1 free parameters · 7 axioms · 0 invented entities

The theoretical results rest on two domain assumptions about the missing-data mechanism, the existence of a correction tensor C satisfying (3)-(4), standard convex SGD lemmas, and strong convexity of the objective for the linear-rate result. No new physical entities are introduced, and no fitted parameters enter the central proof.

free parameters (1)

experiment step-size scaling factor = 5000
Hand-chosen in Section 4 to satisfy α < 1/L_g in the synthetic experiments. Not part of the theoretical results, which hold for any α satisfying the bound.

axioms (7)

domain assumption Assumption 1: either each row is chosen at most once, or the mask is redrawn each iteration
Section 2; needed so that the per-iteration expectation of the gradient estimate is the full gradient. The 'at most once' branch is not sufficient for conditional unbiasedness under without-replacement sampling.
domain assumption Assumption 2: every entry of A occurs with marginal probability p (E[˜A]=pE[A])
Section 2; used in Lemma 4 and Proposition 1 to compute expectations of mask products.
domain assumption Existence of a binary (and for Theorem 2, Hermitian) tensor C satisfying (3) and (4) for all rows
Section 2, Equations (3)-(4); the core condition making the gradient estimate unbiased. Constructed for the three models in Proposition 1.
standard math Lemma 1 (Shamir-Zhang): standard SGD convergence for convex F with unbiased, bounded-variance gradients
Section 2, Lemma 1; imported from [20] for Theorem 1.
ad hoc to paper Lemma 2 co-coercivity inequality applied to the random g
Section 3; the stated lemma omits the convexity hypothesis required for the inequality. The f constructed in Lemma 8 can be non-convex, so the inequality is not generally available.
domain assumption F is µ-strongly convex, with µ=σ_min²/m from the T-SVD of A
Section 3, Lemma 7; required for Theorem 2's linear rate; only holds when A is tall and has full t-product column rank.
domain assumption W is a bounded closed convex set containing X⋆
Section 1.2 and Section 2; needed for Lemma 1 to supply K and R constants.

pith-pipeline@v1.3.0-alltime-deepseek · 17262 in / 31892 out tokens · 267032 ms · 2026-08-04T10:56:14.879716+00:00 · methodology

0 comments

read the original abstract

Solving large tensor linear systems poses significant challenges due to the high volume of data stored, and it only becomes more challenging when some of the data is missing. Recently, Ma et al. showed that this problem can be tackled using a stochastic gradient descent-based method, assuming that the missing data follows a uniform missing pattern. We adapt the technique by modifying the update direction, showing that the method is applicable under other missing data models. We prove convergence results and experimentally verify these results on synthetic data.

discussion (0)

Reference graph

Works this paper leans on

24 extracted references · 3 canonical work pages

[1]

In: AAAI 2025 Workshop CoLoRAI (2025)

Ma, A., Needell, D., Xue, A.: Stochastic gradient descent on tensors with missing data. In: AAAI 2025 Workshop CoLoRAI (2025). To appear

2025
[2]

In: Proceedings of the 29th International Conference on Neural Information Processing Systems - Volume 1

Novikov, A., Podoprikhin, D., Osokin, A., Vetrov, D.: Tensorizing neural networks. In: Proceedings of the 29th International Conference on Neural Information Processing Systems - Volume 1. NIPS’15, pp. 442–450. MIT Press, Cambridge, MA, USA (2015)

2015
[3]

In: Proceedings of the Fourth ACM Conference on Recommender Systems

Karatzoglou, A., Amatriain, X., Baltrunas, L., Oliver, N.: Multiverse recommendation: n- dimensional tensor factorization for context-aware collaborative filtering. In: Proceedings of the Fourth ACM Conference on Recommender Systems. RecSys ’10, pp. 79–86. Association for Computing Machinery, New York, NY, USA (2010). https://doi.org/10.1145/1864708.18647...

arXiv 2010
[4]

Linear Algebra and its Applications435(3), 641–658 (2011) https://doi.org/10.1016/j.laa.2010.09.020

Kilmer, M.E., Martin, C.D.: Factorization strategies for third-order tensors. Linear Algebra and its Applications435(3), 641–658 (2011) https://doi.org/10.1016/j.laa.2010.09.020 . Special Issue: Dedication to Pete Stewart on the occasion of his 70th birthday

doi:10.1016/j.laa.2010.09.020 2011
[5]

Numerical Linear Algebra with Applications30(2), 2470 (2023)

Tang, L., Yu, Y., Zhang, Y., Li, H.: Sketch-and-project methods for tensor linear systems. Numerical Linear Algebra with Applications30(2), 2470 (2023)

2023
[6]

arXiv preprint arXiv:2408.13547 (2024) 13

Luo, H., Ma, A.: Frontal slice approaches for tensor linear systems. arXiv preprint arXiv:2408.13547 (2024) 13

Pith/arXiv arXiv 2024
[7]

arXiv preprint arXiv:2412.10583 (2024)

Castillo, A., Haddock, J., Hartsock, I., Hoyos, P., Kassab, L., Kryshchenko, A., Larripa, K., Needell, D., Suryanarayanan, S., Yacoubou-Djima, K.: Randomized kaczmarz methods for t-product tensor linear systems with factorized operators. arXiv preprint arXiv:2412.10583 (2024)

Pith/arXiv arXiv 2024
[8]

IEEE Transactions on Neural Networks and Learning Systems30(3), 851–864 (2019) https://doi.org/10.1109/TNNLS.2018.2851444

Yin, M., Gao, J., Xie, S., Guo, Y.: Multiview subspace clustering via tensorial t-product repre- sentation. IEEE Transactions on Neural Networks and Learning Systems30(3), 851–864 (2019) https://doi.org/10.1109/TNNLS.2018.2851444

arXiv 2019
[9]

https://arxiv.org/abs/1811.06569

Newman, E., Horesh, L., Avron, H., Kilmer, M.: Stable Tensor Neural Networks for Rapid Deep Learning (2018). https://arxiv.org/abs/1811.06569

Pith/arXiv arXiv 2018
[10]

IEEE Transactions on Neural Networks and Learning Systems28(12), 2961–2973 (2017) https: //doi.org/10.1109/TNNLS.2016.2611525

Hu, W., Tao, D., Zhang, W., Xie, Y., Yang, Y.: The twist tensor nuclear norm for video completion. IEEE Transactions on Neural Networks and Learning Systems28(12), 2961–2973 (2017) https: //doi.org/10.1109/TNNLS.2016.2611525

arXiv 2017
[11]

Numeri- cal Mathematics: Theory, Methods and Applications12(1), 1–20 (2018) https://doi.org/10.4208/ nmtma.OA-2018-0066

Ma, A., Needell, D.: Stochastic gradient descent for linear systems with missing data. Numeri- cal Mathematics: Theory, Methods and Applications12(1), 1–20 (2018) https://doi.org/10.4208/ nmtma.OA-2018-0066

2018
[12]

IEEE Transactions on Signal Processing68, 2155–2169 (2020)

Amiri, M.M., G¨ und¨ uz, D.: Machine learning at the wireless edge: Distributed stochastic gradient descent over-the-air. IEEE Transactions on Signal Processing68, 2155–2169 (2020)

2020
[13]

In: Proceedings of COMP- STAT’2010: 19th International Conference on Computational StatisticsParis France, August 22-27, 2010 Keynote, Invited and Contributed Papers, pp

Bottou, L.: Large-scale machine learning with stochastic gradient descent. In: Proceedings of COMP- STAT’2010: 19th International Conference on Computational StatisticsParis France, August 22-27, 2010 Keynote, Invited and Contributed Papers, pp. 177–186 (2010). Springer

2010
[14]

In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

Maehara, T., Hayashi, K., Kawarabayashi, K.-i.: Expected tensor decomposition with stochastic gradient descent. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30(1) (2016)

2016
[15]

SIAM Journal on Mathematics of Data Science2(4), 1066–1095 (2020)

Kolda, T.G., Hong, D.: Stochastic gradients for large-scale tensor decomposition. SIAM Journal on Mathematics of Data Science2(4), 1066–1095 (2020)

2020
[16]

In: 2017 IEEE International Conference on Big Data (Big Data), pp

Papastergiou, T., Megalooikonomou, V.: A distributed proximal gradient descent method for tensor completion. In: 2017 IEEE International Conference on Big Data (Big Data), pp. 2056–2065 (2017). https://doi.org/10.1109/BigData.2017.8258152

arXiv 2017
[17]

SIAM Jour- nal on Imaging Sciences14(4), 1439–1471 (2021) https://doi.org/10.1137/21M1398562 https://doi.org/10.1137/21M1398562

Chen, X., Qin, J.: Regularized kaczmarz algorithms for tensor recovery. SIAM Jour- nal on Imaging Sciences14(4), 1439–1471 (2021) https://doi.org/10.1137/21M1398562 https://doi.org/10.1137/21M1398562

work page doi:10.1137/21m1398562 2021
[18]

BIT Numerical Mathematics 62(1), 171–194 (2022) https://doi.org/10.1007/s10543-021-00877-w

Ma, A., Molitor, D.: Randomized kaczmarz for tensor linear systems. BIT Numerical Mathematics 62(1), 171–194 (2022) https://doi.org/10.1007/s10543-021-00877-w

work page doi:10.1007/s10543-021-00877-w 2022
[19]

Inverse Problems and Imaging18(4), 889–907 (2024) https://doi.org/10.3934/ ipi.2023059

Grotheer, R., Li, S., Ma, A., Needell, D., Qin, J.: Iterative singular tube hard thresholding algorithms for tensor recovery. Inverse Problems and Imaging18(4), 889–907 (2024) https://doi.org/10.3934/ ipi.2023059

2024
[20]

Shamir, O., Zhang, T.: Stochastic gradient descent for non-smooth optimization: Convergence results and optimal averaging schemes.Proc. Int. Conf. Machine Learning, 71–79 (2013)

2013
[21]

Advances in neural information processing systems27(2014)

Needell, D., Ward, R., Srebro, N.: Stochastic gradient descent, weighted sampling, and the randomized kaczmarz algorithm. Advances in neural information processing systems27(2014)

2014
[22]

https://arxiv.org/abs/2503.19155

Castillo, A., Haddock, J., Hartsock, I., Hoyos, P., Kassab, L., Kryshchenko, A., Larripa, K., Needell, D., Suryanarayanan, S., Djima, K.Y.: Block Gauss-Seidel methods for t-product tensor regression (2025). https://arxiv.org/abs/2503.19155

Pith/arXiv arXiv 2025
[23]

He, H., Ling, C., Xie, W.: Tensor completion via a generalized transformed tensor t-product 14 decomposition without t-SVD. J. Sci. Comput.93(2) (2022)

2022
[24]

Kernfeld, E., Kilmer, M., Aeron, S.: Tensor–tensor products with invertible linear transforms. Linear Algebra and its Applications485, 545–570 (2015) https://doi.org/10.1016/j.laa.2015.07.021 6 Appendix Lemma 3 proves a straightforward sub-multiplicative property of the Frobenius norm. Lemma 4 gives a bound onE[∥g(X)∥ 2], Lemma 5 gives a bound onE[∥g(X ⋆)...

work page doi:10.1016/j.laa.2015.07.021 2015

[1] [1]

In: AAAI 2025 Workshop CoLoRAI (2025)

Ma, A., Needell, D., Xue, A.: Stochastic gradient descent on tensors with missing data. In: AAAI 2025 Workshop CoLoRAI (2025). To appear

2025

[2] [2]

In: Proceedings of the 29th International Conference on Neural Information Processing Systems - Volume 1

Novikov, A., Podoprikhin, D., Osokin, A., Vetrov, D.: Tensorizing neural networks. In: Proceedings of the 29th International Conference on Neural Information Processing Systems - Volume 1. NIPS’15, pp. 442–450. MIT Press, Cambridge, MA, USA (2015)

2015

[3] [3]

In: Proceedings of the Fourth ACM Conference on Recommender Systems

Karatzoglou, A., Amatriain, X., Baltrunas, L., Oliver, N.: Multiverse recommendation: n- dimensional tensor factorization for context-aware collaborative filtering. In: Proceedings of the Fourth ACM Conference on Recommender Systems. RecSys ’10, pp. 79–86. Association for Computing Machinery, New York, NY, USA (2010). https://doi.org/10.1145/1864708.18647...

arXiv 2010

[4] [4]

Linear Algebra and its Applications435(3), 641–658 (2011) https://doi.org/10.1016/j.laa.2010.09.020

Kilmer, M.E., Martin, C.D.: Factorization strategies for third-order tensors. Linear Algebra and its Applications435(3), 641–658 (2011) https://doi.org/10.1016/j.laa.2010.09.020 . Special Issue: Dedication to Pete Stewart on the occasion of his 70th birthday

doi:10.1016/j.laa.2010.09.020 2011

[5] [5]

Numerical Linear Algebra with Applications30(2), 2470 (2023)

Tang, L., Yu, Y., Zhang, Y., Li, H.: Sketch-and-project methods for tensor linear systems. Numerical Linear Algebra with Applications30(2), 2470 (2023)

2023

[6] [6]

arXiv preprint arXiv:2408.13547 (2024) 13

Luo, H., Ma, A.: Frontal slice approaches for tensor linear systems. arXiv preprint arXiv:2408.13547 (2024) 13

Pith/arXiv arXiv 2024

[7] [7]

arXiv preprint arXiv:2412.10583 (2024)

Castillo, A., Haddock, J., Hartsock, I., Hoyos, P., Kassab, L., Kryshchenko, A., Larripa, K., Needell, D., Suryanarayanan, S., Yacoubou-Djima, K.: Randomized kaczmarz methods for t-product tensor linear systems with factorized operators. arXiv preprint arXiv:2412.10583 (2024)

Pith/arXiv arXiv 2024

[8] [8]

IEEE Transactions on Neural Networks and Learning Systems30(3), 851–864 (2019) https://doi.org/10.1109/TNNLS.2018.2851444

Yin, M., Gao, J., Xie, S., Guo, Y.: Multiview subspace clustering via tensorial t-product repre- sentation. IEEE Transactions on Neural Networks and Learning Systems30(3), 851–864 (2019) https://doi.org/10.1109/TNNLS.2018.2851444

arXiv 2019

[9] [9]

https://arxiv.org/abs/1811.06569

Newman, E., Horesh, L., Avron, H., Kilmer, M.: Stable Tensor Neural Networks for Rapid Deep Learning (2018). https://arxiv.org/abs/1811.06569

Pith/arXiv arXiv 2018

[10] [10]

IEEE Transactions on Neural Networks and Learning Systems28(12), 2961–2973 (2017) https: //doi.org/10.1109/TNNLS.2016.2611525

Hu, W., Tao, D., Zhang, W., Xie, Y., Yang, Y.: The twist tensor nuclear norm for video completion. IEEE Transactions on Neural Networks and Learning Systems28(12), 2961–2973 (2017) https: //doi.org/10.1109/TNNLS.2016.2611525

arXiv 2017

[11] [11]

Numeri- cal Mathematics: Theory, Methods and Applications12(1), 1–20 (2018) https://doi.org/10.4208/ nmtma.OA-2018-0066

Ma, A., Needell, D.: Stochastic gradient descent for linear systems with missing data. Numeri- cal Mathematics: Theory, Methods and Applications12(1), 1–20 (2018) https://doi.org/10.4208/ nmtma.OA-2018-0066

2018

[12] [12]

IEEE Transactions on Signal Processing68, 2155–2169 (2020)

Amiri, M.M., G¨ und¨ uz, D.: Machine learning at the wireless edge: Distributed stochastic gradient descent over-the-air. IEEE Transactions on Signal Processing68, 2155–2169 (2020)

2020

[13] [13]

In: Proceedings of COMP- STAT’2010: 19th International Conference on Computational StatisticsParis France, August 22-27, 2010 Keynote, Invited and Contributed Papers, pp

Bottou, L.: Large-scale machine learning with stochastic gradient descent. In: Proceedings of COMP- STAT’2010: 19th International Conference on Computational StatisticsParis France, August 22-27, 2010 Keynote, Invited and Contributed Papers, pp. 177–186 (2010). Springer

2010

[14] [14]

In: Proceedings of the AAAI Conference on Artificial Intelligence, vol

Maehara, T., Hayashi, K., Kawarabayashi, K.-i.: Expected tensor decomposition with stochastic gradient descent. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 30(1) (2016)

2016

[15] [15]

SIAM Journal on Mathematics of Data Science2(4), 1066–1095 (2020)

Kolda, T.G., Hong, D.: Stochastic gradients for large-scale tensor decomposition. SIAM Journal on Mathematics of Data Science2(4), 1066–1095 (2020)

2020

[16] [16]

In: 2017 IEEE International Conference on Big Data (Big Data), pp

Papastergiou, T., Megalooikonomou, V.: A distributed proximal gradient descent method for tensor completion. In: 2017 IEEE International Conference on Big Data (Big Data), pp. 2056–2065 (2017). https://doi.org/10.1109/BigData.2017.8258152

arXiv 2017

[17] [17]

SIAM Jour- nal on Imaging Sciences14(4), 1439–1471 (2021) https://doi.org/10.1137/21M1398562 https://doi.org/10.1137/21M1398562

Chen, X., Qin, J.: Regularized kaczmarz algorithms for tensor recovery. SIAM Jour- nal on Imaging Sciences14(4), 1439–1471 (2021) https://doi.org/10.1137/21M1398562 https://doi.org/10.1137/21M1398562

work page doi:10.1137/21m1398562 2021

[18] [18]

BIT Numerical Mathematics 62(1), 171–194 (2022) https://doi.org/10.1007/s10543-021-00877-w

Ma, A., Molitor, D.: Randomized kaczmarz for tensor linear systems. BIT Numerical Mathematics 62(1), 171–194 (2022) https://doi.org/10.1007/s10543-021-00877-w

work page doi:10.1007/s10543-021-00877-w 2022

[19] [19]

Inverse Problems and Imaging18(4), 889–907 (2024) https://doi.org/10.3934/ ipi.2023059

Grotheer, R., Li, S., Ma, A., Needell, D., Qin, J.: Iterative singular tube hard thresholding algorithms for tensor recovery. Inverse Problems and Imaging18(4), 889–907 (2024) https://doi.org/10.3934/ ipi.2023059

2024

[20] [20]

Shamir, O., Zhang, T.: Stochastic gradient descent for non-smooth optimization: Convergence results and optimal averaging schemes.Proc. Int. Conf. Machine Learning, 71–79 (2013)

2013

[21] [21]

Advances in neural information processing systems27(2014)

Needell, D., Ward, R., Srebro, N.: Stochastic gradient descent, weighted sampling, and the randomized kaczmarz algorithm. Advances in neural information processing systems27(2014)

2014

[22] [22]

https://arxiv.org/abs/2503.19155

Castillo, A., Haddock, J., Hartsock, I., Hoyos, P., Kassab, L., Kryshchenko, A., Larripa, K., Needell, D., Suryanarayanan, S., Djima, K.Y.: Block Gauss-Seidel methods for t-product tensor regression (2025). https://arxiv.org/abs/2503.19155

Pith/arXiv arXiv 2025

[23] [23]

He, H., Ling, C., Xie, W.: Tensor completion via a generalized transformed tensor t-product 14 decomposition without t-SVD. J. Sci. Comput.93(2) (2022)

2022

[24] [24]

Kernfeld, E., Kilmer, M., Aeron, S.: Tensor–tensor products with invertible linear transforms. Linear Algebra and its Applications485, 545–570 (2015) https://doi.org/10.1016/j.laa.2015.07.021 6 Appendix Lemma 3 proves a straightforward sub-multiplicative property of the Frobenius norm. Lemma 4 gives a bound onE[∥g(X)∥ 2], Lemma 5 gives a bound onE[∥g(X ⋆)...

work page doi:10.1016/j.laa.2015.07.021 2015