{"id":"5f48422c-2406-49e8-8799-994e0e680045","arxiv_id":"2502.01293","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"A tensor-train implementation of LSQR with sketching is introduced for multiterm tensor least squares and tested on text and image classification.","lead":"This paper presents TT-LSQR, a tensor-train version of the classical LSQR iterative solver, for least squares problems with sums of Kronecker-product terms. It applies the solver to document and image classification, reporting promising but small-sample results.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The load-bearing gap is that TT-LSQR's mandatory rounding is never shown to keep iterates close to the true LS minimizer; without a perturbation bound or low-rank representability result, the method may solve a different rank-constrained problem.","rationale":"Read in good faith, the paper's contribution is a practical TT implementation of LSQR with truncation, plus an application. The algorithmic description is coherent and the PDE sanity checks are useful evidence of basic functionality. The reader's conditional verdict identifies the same central weakness I find: the algorithm's defining feature — rounding at every step — is also the place where the link to the original least squares problem is most fragile. I do not see a contradiction inside the algorithm as written, but the absence of any perturbation or convergence analysis for the truncated recurrence means the central claim rests on an unverified premise. A concrete small experiment can settle whether this premise is plausible: if tightening the rounding tolerance drives the TT-LSQR residual to the exact LS residual in a well-conditioned problem, the concern is weakened; if instead the residual floors at the truncation level (as §5 suggests), then the method solves a rank-constrained problem rather than (1.1). Because the reader already conditions acceptance on this kind of evidence, my assessment does not move the verdict; it strengthens the reason for the condition. Secondary issues (tiny 20-document test sets, no released code, mis-stated sketching dimension) reinforce but do not replace this main concern.","tokens_in":17458,"tokens_out":8509,"duration_ms":81330,"concrete_test":"Generate a small overdetermined instance of (1.1) with d=3, ℓ=2, m_j=5, n_j=10, exact solution X* = A^+vec(F), and run the published Algorithm 3.1 with TT rounding tolerances 1e-3, 1e-6, 1e-9 and rank caps 5, 10, 20. Compare the returned X and its residual against exact vectorized LSQR after the same number of iterations. If the relative error does not decrease toward the exact LS residual as the tolerance tightens and rank cap grows (and, separately, if the TT-ranks of X* are not small), then the truncation changes the solved problem rather than merely compressing an accurate iterate.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3 introduces TT-LSQR as tensor LSQR plus rounding, stating that after each sum 'some truncation is necessary' and that iterates match the vectorized case 'before truncation.' Section 3.1 then rounds every updated tensor via TT-SVD; Theorem 3.1 bounds a single rounding step only. No theorem bounds the accumulated effect of repeatedly rounding U_{i+1}, V_{i+1}, and X_i, and no argument shows that the exact minimizer of (1.1) has the low TT-rank the iterates are forced to have. Once rounding is applied, the scalars α_i, β_i, ρ_i, θ_i no longer describe a fixed linear operator, so the standard LSQR convergence theory does not apply: the algorithm may be converging to the solution of a different, tolerance-and-rank-dependent problem. The sanity checks in §5 show exactly this: residual stagnation is controlled by the truncation threshold. The classification experiments in §7 run only 10 iterations with tolerance 1e-4 and never check the returned X against the true least squares residual or against criteria (C1)/(C2) under tighter truncation. The central claim that TT-LSQR effectively solves (1.1) is therefore not established.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TT-LSQR, a tensor-train implementation of the classical LSQR algorithm for the multiterm tensor least squares problem (1.1), where the coefficient is a sum of Kronecker products of tall matrices. The algorithm applies LSQR recurrences to tensors in TT format and rounds the TT-ranks after each sum. The authors also introduce a sketching variant, discuss preconditioning, and apply the method to query-document classification using two criteria (C1) and (C2) on Reuters, Cranfield, Medline, and FashionMNIST data, comparing against a matrix least squares criterion (C3) and a truncated-SVD projection criterion (C4). The central claim is that TT-LSQR provides an effective, memory-saving solver for (1.1) and that the resulting classification framework is promising.","tokens_in":17740,"tokens_out":4619,"duration_ms":45890,"significance":"If the main claim were fully established, the paper would contribute a practical TT-based solver for a genuinely hard class of multiterm tensor least squares problems, with a plausible application to text and image classification. The algorithmic idea is natural, the use of the TT-Toolbox makes the implementation concrete, and the sanity checks in Section 5 usefully document the effect of rounding and rank truncation. The classification criteria are simple and do not involve fitted parameters beyond the least squares solution. However, the absence of any accumulated perturbation analysis for the repeated rounding in the LSQR recurrence leaves the central numerical claim unproven, and the classification evidence rests on very small test sets. The contribution is therefore best regarded as a promising heuristic numerical study rather than a fully supported solver.","major_comments":[{"comment":"The central load-bearing point is that TT-LSQR solves the original problem (1.1), but the paper only states in Section 3 that after each sum 'some truncation is necessary' and that the iterates coincide with the vectorized case only 'before truncation.' No result controls the cumulative effect of rounding U_{i+1}, V_{i+1}, and X_i at every iteration; Theorem 3.1 bounds only a single TT-SVD rounding step. Once rounding is applied, the scalars alpha_i, beta_i, rho_i, and theta_i no longer describe a fixed linear operator, so the classical LSQR convergence theory of [32] does not apply, and the method may converge to a tolerance- and rank-dependent problem rather than to the minimizer of (1.1). Please provide an accumulated perturbation bound, a low-rank representability condition on the true minimizer, or explicitly reframe the contribution as a heuristic rank-constrained solver with empirical validation only.","section":"Section 3, Algorithm 3.1"},{"comment":"The sanity check on the discretized PDE reports only relative residual norms and shows that stagnation is controlled by the truncation threshold. This does not establish that the computed iterates are close to the true solution of (5.1) or to the least squares minimizer of the original problem, since the residual may stagnate at a nonzero value for a different rounded problem. Please include an error measure against the exact solution or against an unconverged reference solution of the full problem, and report the TT-ranks achieved during the iterations.","section":"Section 5, Figure 5.1"},{"comment":"The classification experiments use 20 test documents per category (40 for FashionMNIST), and the authors themselves note that a 5% difference is a single document. Consequently, many of the reported differences between criteria (C1) and (C2) or between values of bar m are not statistically meaningful, and several claims such as 'a general preference of criterion (C2) over (C1)' are not supported at this granularity. Please provide confidence intervals, repeated train/test splits, or larger test sets, and restrict the qualitative conclusions to differences that survive this uncertainty.","section":"Section 7.2, Tables 7.1-7.4"},{"comment":"The sketching construction is not fully specified. If bA_j^{(i)} = S(A_j^{(i)}) with S: R^n -> R^s, then the sketched Kronecker factor has s^d rows, while S(f) as defined maps a single mode to length s; the notation in (4.1) is therefore ambiguous unless S is applied modewise to the whole tensor F. In addition, the chosen sketch dimension s = 2d bar m is far below the s ~ n log n / epsilon^2 required by the cited Johnson-Lindenstrauss bound, so that bound does not justify the reported accuracy. Please define the action of S on f explicitly, justify the sketch dimension, and report the relative residual of the original (unsketched) problem for the sketched solutions.","section":"Section 4.2, Eq. (4.1)"}],"minor_comments":[{"comment":"The sentence 'the conclusions are drown' contains a typo; it should read 'drawn.'","section":"Section 8"},{"comment":"The notation in Criterion (C1), in particular the expression '1 (x)_j f (x) 1' and the accompanying footnote, is unclear as printed; please define the positioning of f in the Kronecker product with a concrete formula.","section":"Section 7.1, Criterion (C1)"},{"comment":"The theorem is cited as [30], which is the TT-cross paper by Oseledets and Tyrtyshnikov, whereas the stated TT-SVD error estimate is usually attributed to [28]; please verify and correct the citation.","section":"Section 3.1, Theorem 3.1"},{"comment":"For the case ell = 3 the table reports results on only 10 test documents while the other rows use 20 documents; this difference should be stated directly in the table caption or in the surrounding text, since it affects the interpretability of the CPU-time comparison.","section":"Table 7.5"},{"comment":"The text says the code 'will be made available' at a GitHub repository, but no repository link or release is provided; please include the link or state clearly that the code is available upon request.","section":"Section 3, code availability"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is a reasonable numerical study of a heuristic tensor LSQR variant, but the central theoretical gap—no accumulated perturbation bound for the rounded recurrence—is significant for a numerical analysis journal. I would advise the editor to require either a genuine error analysis or an explicit reframing of the paper as an empirical study of a rank-constrained heuristic, together with statistically stronger classification experiments, before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: TT-LSQR is a natural and clearly written tensor-train adaptation of LSQR for the multiterm tensor least squares problem (1.1), but the central property that the truncated recurrence still tracks the original problem is unproven. I would send it to referees, mainly to force that question.\n\nWhat is new: this is the first TT-format LSQR for the sum-of-Kronecker-products setting, and it adds randomized sketching and a document/image classification study. The writing is clear, the algorithm is easy to implement with the TT-Toolbox, and the authors are honest about the small classification samples, explicitly noting that 5% equals one document. The preconditioning and sketching discussion is sensible, and using the sketched solution as a starting guess for a couple of unsketched iterations is a reasonable practical trick.\n\nThe soft spots are serious but not fatal. Section 3 introduces rounding as necessary, but the only error bound cited (Theorem 3.1) is for a single TT-SVD step. Once you round U_{i+1}, V_{i+1}, and X_i every iteration, the scalars alpha, beta, rho, and theta no longer describe a fixed linear operator, so standard LSQR convergence theory does not apply. The sanity check in Figure 5.1 shows residual stagnation at the truncation threshold; that is consistent with converging to a different, tolerance-dependent rank-constrained problem, not with solving (1.1). The classification experiments run only 10 iterations with tolerance 1e-4 and never check the returned X against the true least squares residual or against tighter truncation. The sample sizes are 20 to 40 test documents with no confidence intervals, so the percentage differences between methods are often within noise. The sketching dimension is set to s = 2d m-bar, far below the s ~ n log n / eps^2 needed for the quoted theoretical bound; the start-sketch strategy is sensible but not analyzed. No code is shipped yet, so the numerical results are not independently reproducible.\n\nNone of this means the method is wrong. It may work well in practice, and the classification results are intriguing. But the paper currently supports an algorithmic proposal, not a reliable solver claim.\n\nWho is this for? Numerical tensor researchers and data mining practitioners who want a heuristic tool for (1.1) and are willing to treat it as preliminary. It is a legitimate extension of the authors' matrix LSQR program, with honest citation of prior work and no circularity.\n\nRecommendation: accept for peer review, likely as a major revision. Ask for either a perturbation bound on the truncated recurrence or a backward-error-style argument, and stronger empirical validation: more test documents, confidence intervals, residual checks, and a comparison of the truncated solution against the true LS residual. With that, the paper becomes useful; without it, the central claim remains unestablished.","headline":"A plausible TT-LSQR extension whose truncation step is never shown to track the original least squares problem; worth refereeing, but the reliability claims need support.","tokens_in":18279,"tokens_out":2568,"would_cite":true,"duration_ms":25780,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["65F45","65F55","15A23"],"pacs":[],"model":"deepseek-v4-flash","headline":"A tensor-train reformulation of LSQR solves multiterm tensor least-squares problems without forming Kronecker products, and the paper shows the resulting solver classifies documents and images competitively.","keywords":["tensor least squares","tensor-train format","LSQR","rank truncation","Kronecker products","randomized sketching","document classification","image classification"],"falsifier":"On a small instance of the multiterm tensor least-squares problem where the unrounded vectorized LSQR minimizer can be computed exactly, run TT-LSQR with no rounding and with a sequence of looser rounding tolerances; if the rounded solution errors do not tend to the unrounded solution as the tolerance tends to zero, or if the residual stops decreasing at a level inconsistent with the tolerance, the central claim fails.","tokens_in":17212,"feed_emoji":"🧮","tokens_out":12411,"duration_ms":106185,"temperature":0.7,"pith_summary":"The paper tries to establish that the multiterm tensor least-squares problem $\\min_{\\mathcal{X}} \\|\\mathcal{F} - \\sum_{i=1}^{\\ell} \\mathcal{X} \\times_1 A_1^{(i)} \\cdots \\times_d A_d^{(i)}\\|_F$ can be solved by a tensor-train implementation of the classic iterative LSQR method, without ever forming the Kronecker-product matrix that vectorization would produce. Its proposal is to run the LSQR recurrence with tensors in place of vectors, storing every iterate in the tensor-train format and truncating ranks after each sum. The authors argue this is practical: memory for the iterates scales with the TT ranks rather than with the full grid size, and a randomized sketch can shrink the tall coefficient matrices. In experiments on three term-document corpora and on Fashion-MNIST images, the resulting solver, used with two tensor-based classification criteria, gives success rates that are competitive with, and for the simple matrix baseline usually better than, standard query-matching and projection methods. A sympathetic reader would take the paper's core claim to be that problems of this form are now accessible to a memory-saving iterative solver, and that this solver is accurate enough to be useful in data mining.","feed_headline":"Tensor-train LSQR solves big multiterm least squares problems","feed_subtitle":"Storing tensors as linked cores sidesteps infeasible Kronecker products and classifies documents and images competitively.","key_machinery":"The load-bearing object is the tensor-train (TT) decomposition, which writes a $d$-dimensional tensor as a chain of small cores, $\\mathcal{X}(i_1,\\dots,i_d)=G_1(i_1)G_2(i_2)\\cdots G_d(i_d)$ with $r_0=r_d=1$, so storage scales with the TT ranks instead of with the full multidimensional grid. The algorithm that carries the argument is the LSQR recurrence itself: vectors become tensors, the coefficient operator applies the sum of Kronecker products mode by mode without forming the products explicitly, and TT-SVD rounding after each sum keeps the ranks bounded. Two accelerants complete the machinery: a preconditioner built from the upper-triangular QR factors of selected coefficient matrices, and a randomized sketching operator that projects each mode's matrix onto a much smaller row space, approximately preserving Euclidean distances, before the iteration starts.","core_discovery":"On its own terms, the central discovery is that the LSQR recurrence is format-agnostic: replacing the vectors by tensors and the operator by $\\mathcal{L}(\\mathcal{X})=\\sum_i \\mathcal{X} \\times_1 A_1^{(i)} \\cdots \\times_d A_d^{(i)}$ yields exactly the same iterates as the vectorized method before any rounding is applied. Because sums of tensor-train tensors have ranks equal to the sum of the addend ranks, the paper applies TT-SVD rounding after each addition and application of $\\mathcal{L}$, at the cost of a per-step error controlled by the rounding tolerance. It then shows that this truncated tensor LSQR, optionally preceded by QR-based right preconditioning and applied to a randomized sketch of each mode, solves the multiterm problem and produces classification decisions through two criteria: a direct residual criterion and a subspace-projection criterion built from the singular vectors of the TT-cores. The paper's claim is that this combination is a workable solver and a competitive tool for allocating new documents or images to existing clusters.","pith_inferences":["Inference: The format-level nature of the rounding suggests the same tensorized template should transfer to other Krylov iterations, such as CG on the normal equations or GMRES, for sums of Kronecker products; the paper only develops LSQR.","Inference: The authors mention combining the two classification criteria as future work; a natural test is a voting rule that could stabilize classification on weakly correlated groups such as the Fashion-MNIST sandals class.","Inference: The sketch is applied per mode with the same sketch size; one could test mode-dependent sketch sizes, since the reported correlation patterns differ markedly across groups and datasets."],"forward_implications":["Multiterm tensor least-squares problems with three or more modes become solvable without forming or storing the Kronecker-product coefficient matrix, whose size would be infeasible.","Truncation gives a controllable trade-off: looser TT rounding tolerances or smaller rank caps make the iterates stagnate at predictable residual levels, matching the behavior known for truncated matrix Krylov methods.","For fixed total training data, splitting it into more terms $\\ell$ with fewer columns $m$ per term lowers CPU time dramatically while keeping classification accuracy roughly constant.","Sketching the keyword or pixel space cuts CPU time by a large factor, and a short unsketched refinement using the sketched solution as a starting guess often recovers most of the lost accuracy.","The tensor-based subspace-projection criterion (C2) generally matches or beats simple matrix least-squares query matching, which is unreliable on weakly correlated clusters."],"supporting_citations":[{"why":"Supplies the vector LSQR recurrence that the tensor algorithm mirrors.","marker":"[32]"},{"why":"Defines the tensor-train decomposition and the TT-SVD rounding used at every update.","marker":"[28]"},{"why":"Provides the software routines that implement TT sums, products, and rounding throughout the iteration.","marker":"[29]"},{"why":"Extends LSQR to tensor least squares for the single-term case, the template the paper generalizes to multiple terms.","marker":"[2]"},{"why":"Gives the truncated matrix-oriented LSQR for a two-term matrix problem, the closest predecessor of the tensor version.","marker":"[34]"},{"why":"Supplies the randomized sketching bounds used to justify replacing the full problem by a projected one.","marker":"[26]"},{"why":"Contributes the idea of using the sketched solution as a starting guess for a few full-problem iterations.","marker":"[35]"},{"why":"Frames query classification as a least-squares problem and provides the projection baseline used for comparison.","marker":"[14]"},{"why":"Gives the TT-SVD error estimate quoted to justify rounding in the tensor-train format.","marker":"[30]"}],"fun_headline_variants":["TT-LSQR: tensor-train LSQR sidesteps Kronecker blowup","LSQR is format-agnostic; tensor-train cores store big tensors","TT-LSQR with sketching: fast tall-matrix tensor solves","Tensor-train LSQR classifies documents via residual or projection"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method rests on the assumption that repeatedly compressing the tensor-train representation after each update does not carry the iterates away from the true least-squares solution; the paper validates the per-step truncation error but does not analyze how these errors accumulate over hundreds of iterations.","fun_headline_variants_meta":{"raw":{"variants":["TT-LSQR: tensor-train LSQR sidesteps Kronecker blowup","LSQR is format-agnostic; tensor-train cores store big tensors","TT-LSQR with sketching: fast tall-matrix tensor solves","Tensor-train LSQR classifies documents via residual or projection"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000647,"raw_usage":{"total_tokens":2987,"prompt_tokens":975,"completion_tokens":2012,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":591,"completion_tokens_details":{"reasoning_tokens":1930}},"tokens_in":591,"tokens_out":2012,"duration_ms":15500,"temperature":1.0,"reasoning_tokens":1930,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T15:47:07.444372+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"On a small instance of the multiterm tensor least-squares problem where the unrounded vectorized LSQR minimizer can be computed exactly, run TT-LSQR with no rounding and with a sequence of looser rounding tolerances; if the rounded solution errors do not tend to the unrounded solution as the tolerance tends to zero, or if the residual stops decreasing at a level inconsistent with the tolerance, the central claim fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the vector LSQR recurrence that the tensor algorithm mirrors."},{"cited_title":"Oseledets, Tensor-train decomposition, SIAM J","cited_arxiv_id":null,"evidence_quote":"Defines the tensor-train decomposition and the TT-SVD rounding used at every update."},{"cited_title":"Oseledets, TT-Toolbox, 2025","cited_arxiv_id":null,"evidence_quote":"Provides the software routines that implement TT sums, products, and rounding throughout the iteration."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Extends LSQR to tensor least squares for the single-term case, the template the paper generalizes to multiple terms."},{"cited_title":"Piccinini and V","cited_arxiv_id":null,"evidence_quote":"Gives the truncated matrix-oriented LSQR for a two-term matrix problem, the closest predecessor of the tensor version."},{"cited_title":"Martinsson and J","cited_arxiv_id":null,"evidence_quote":"Supplies the randomized sketching bounds used to justify replacing the full problem by a projected one."},{"cited_title":"Rokhlin and M","cited_arxiv_id":null,"evidence_quote":"Contributes the idea of using the sketched solution as a starting guess for a few full-problem iterations."},{"cited_title":"Eld ´en, Matrix methods in Data Mining and Pattern Recognition , Fundamentals of Algo- rithms, SIAM, Philadelphia, USA, 2019","cited_arxiv_id":null,"evidence_quote":"Frames query classification as a least-squares problem and provides the projection baseline used for comparison."},{"cited_title":"Oseledets and E","cited_arxiv_id":null,"evidence_quote":"Gives the TT-SVD error estimate quoted to justify rounding in the tensor-train format."}],"review_version":1}