Tensor Topic Modeling Via HOSVD

Claire Donnat; Yating Liu

REVIEW 2 major objections 4 minor 101 references

Tensor Topic Modeling Via HOSVD

T0 review · 2 major / 4 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read HOSVD gives tensor topic models their first consistent core estimator.

desk verdict A workmanlike HOSVD extension of matrix pLSI that delivers a genuinely new consistent core-tensor estimator; the scary circular proof line is a typo, and the real weakness is the three-way anchor assumption. read the letter →

arxiv 2501.00535 v1 pith:WCP5HIDZ submitted 2024-12-31 math.ST stat.APstat.TH

Yating Liu , Claire Donnat This is my paper · ORCID

classification math.STstat.APstat.TH MSC 62H3062H2562G2015A69

keywords TuckerdecompositionHOSVDtopicmodelingpLSIanchorwordvertexhuntingcoretensorentrywiseℓ1error

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper extends probabilistic latent semantic indexing (pLSI) from document-term matrices to three-way count tensors, modeling the expected corpus as a nonnegative Tucker decomposition with a small core tensor and three factor matrices: reviewer types, paper categories, and word topics. The authors propose a modified higher-order singular value decomposition (HOSVD) estimator, combined with SCORE normalization and vertex hunting, to recover all four components, and they prove that the core tensor $\mathcal{G}$ is estimated consistently in this setting. Under anchor conditions on all three modes and a minimum word-frequency bound, the estimator achieves explicit entry-wise $\ell_1$ rates that shrink as the number of documents $N^{(1)}N^{(2)}$ and the document length $M$ grow. A sparse variant based on column-wise $\ell_q$ sparsity handles large vocabularies with rare words. If the result holds, tensor topic modeling becomes a statistically grounded way to handle longitudinal texts, microbiome surveys, and market-basket data, including estimates of the interactions among latent clusters.

What carries the argument

The machinery is the mode-wise matricization of the tensor together with the higher-order singular value decomposition (HOSVD), which is used to estimate low-dimensional subspaces along each mode. Under anchor assumptions, the rows of the leading singular-vector matrix of each matricization form an ideal simplex, so a vertex-hunting algorithm such as successive projection identifies the vertices; for the word mode, SCORE normalization turns the point cloud into an ideal simplex. The factor matrices are read off as the resulting mixture weights, and the core tensor is recovered through the identity $\mathcal{G} = \mathcal{S} \cdot (\tilde V^{(1)}, \tilde V^{(2)}, \tilde V^{(3)})$, where $\mathcal{S}$ is the HOSVD core and each $\tilde V^{(a)}$ is the corresponding vertex matrix. This last step is the piece that avoids flattening the tensor and gives the first consistent core estimate.

What would settle it

Run the estimator on simulated corpora generated from a well-conditioned Tucker topic model in which one topic has no anchor word but all rows of $A^{(3)}$ have overlapping support; the vertex-hunting step should fail to recover the true vertices and the empirical $\ell_1$ error for $A^{(3)}$ should stay bounded away from the theorem's rate as the tensor dimensions grow.

Watch

Extended reading notes

Core claim

The central claim is that a HOSVD-based pipeline recovers all components of a Tucker-parameterized tensor topic model at provable rates. For the factor matrices, $\mathcal{L}(\hat A^{(1)}, A^{(1)}) \leq c^* N^{(1)} \sqrt{\log(N_R)/M}$ and $\mathcal{L}(\hat A^{(2)}, A^{(2)}) \leq c^* N^{(2)} \sqrt{\log(N_R)/M}$, while the word-topic matrix satisfies $\mathcal{L}(\hat A^{(3)}, A^{(3)}) \leq c^* (\log(N_R)/(N^{(1)}N^{(2)}M))^{1/4}$, and the core tensor $\mathcal{G}$ satisfies $\mathcal{L}(\hat{\mathcal{G}}, \mathcal{G}) \leq c^* \max\{\sqrt{\log(N_R)/M}, (\log(N_R)/(N^{(1)}N^{(2)}M))^{1/4}\}$, all with probability at least $1 - o(N_R^{-1})$. The paper claims this is the first consistent estimator of the core tensor in Tucker-based tensor topic modeling, and it verifies the behavior on synthetic, scientific-abstract, microbiome, and market-basket data.

Load-bearing premise

The whole argument depends on an anchor condition that must hold in all three modes simultaneously — every topic needs a pure anchor word, every reviewer cluster a pure anchor reviewer, and every paper category a pure anchor paper — together with a minimum word-frequency bound.

Editorial extensions

If this is right

For a corpus of $N^{(1)} \times N^{(2)}$ documents over $R$ words with document length $M$, the factor matrices are estimated at rates that vanish whenever $M$ grows faster than $\log(N_R)$, so tensor topic estimation is consistent in the same regime as matrix pLSI.
The core tensor $\mathcal{G}$ is recovered at essentially the same statistical cost as the factors, meaning that interactions among reviewer types, paper categories, and topics can be inferred directly rather than by post-processing.
Under the column-wise $\ell_q$ sparsity assumption, low-frequency words can be thresholded and set to zero without changing the rates for $A^{(1)}$, $A^{(2)}$, or $\mathcal{G}$, which makes the method usable for large vocabularies.
Because the estimator is spectral and deterministic given the vertex-hunting step, the pipeline gives reproducible estimates and scales more cheaply than Bayesian tensor-LDA analogues.
The method applies not only to text but to any count tensor with a multinomial sampling model, including longitudinal microbiome samples and customer purchase baskets over time.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The paper proves consistency under anchor conditions on every mode; a natural untested extension would be a one-mode anchor setting or a relaxation via archetypal analysis, where the same HOSVD pipeline would require a modified simplex step.
The core-tensor consistency result immediately suggests a hypothesis test for mode interactions: fit the Tucker model on training data and compare estimated core slices across $k^{(2)}$ to decide whether paper category changes the topic mix for a reviewer type.
Because the minimum-frequency condition is the tightest bottleneck, the sparse thresholded variant is likely the relevant operating regime for real vocabularies; one could benchmark the dense theorem versus the sparse variant by varying $R$ with fixed $N^{(1)}N^{(2)}M$.
The HOSVD core formula can be coupled with higher-order orthogonal iterations, and the paper's own comparison suggests that would preserve the rates under sub-Gaussian noise, although that combined estimator is not implemented here.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Bottom line: this is a real frequentist estimator for tensor topic models, and the core-tensor part is genuinely new. The factor matrix steps are standard matrix pLSI machinery applied mode by mode via HOSVD, but the V-tilde identity that lets them recover G from the HOSVD core is a clean and useful idea. The paper is honest about its assumptions, and the appendix actually contains the proof of Lemma D.8; the self-referential sentence after it is an editing artifact, not a missing argument. The derivation on the page above bounds (49)-(51) correctly from Lemmas D.6 and D.7. So the stress-test objection, while it flags a sloppy line, does not land as a load-bearing flaw.

What is new: first consistency guarantee for the core tensor in Tucker-based tensor topic models, with rates in entrywise l1. The rates for A(1), A(2) match the matrix minimax results up to logs, and A(3) gets the expected 1/4-power rate. The sparsity adaptation is a practical add-on. Experiments are extensive, cover synthetic plus three real datasets, and the code is public. The comparisons with Tensor-LDA, STM, NTD, and hybrid methods are honest, even when the proposed method wins.

Soft spots: the anchor conditions on all three modes are genuinely restrictive, and the paper says so. Requiring pure reviewers, pure papers, and pure words simultaneously is much stronger than the single anchor dimension needed for matrix identifiability. The min-frequency condition in Theorem 2.4 is unrealistic for large vocabularies; Assumption 2.4 mitigates it but only under the Zipf-like decay. These are real limitations, though not hidden ones. The HOOI remark (Remark 2.3) is underdeveloped: it claims comparable bounds but the lemma it points to does not really analyze HOOI in the multinomial case. The real-data evaluation is more exploratory than confirmatory; I would not take the Dirichlet regression p-values too literally.

Who gets value: researchers in topic modeling, tensor decompositions, and spectral estimation of latent variable models. It is also a useful reference for practitioners who want a fast deterministic alternative to Bayesian tensor LDA.

Peer review: yes, send it out. The central result is likely correct, the proof skeleton is there, and the core-tensor estimator fills a real gap. I would ask the authors to delete the circular sentence, expand the HOOI comparison, and maybe add a failure simulation where the anchor conditions break. Those are revisions, not grounds for rejection.

Referee Report

2 major / 4 minor

Summary. The paper introduces Tensor Topic Modeling (TTM), a pLSI-type model in which an order-3 tensor of word counts has expectation D = G · (A(1), A(2), A(3)) under a nonnegative Tucker decomposition. The proposed procedure uses HOSVD on the three matricizations, recovers A(1) and A(2) by ideal-simplex vertex hunting, recovers A(3) via SCORE normalization followed by vertex hunting, and then reconstructs the core tensor G from the estimated singular subspaces and vertices. Theorem 2.4 gives high-probability entry-wise ℓ1 rates for all three factor matrices and the core tensor under anchor, eigengap, and minimum word-frequency assumptions, and Lemma 2.2 extends the A(3) rate to a weak column-wise ℓq sparsity setting. The paper reports synthetic experiments and three real-data applications (arXiv abstracts, vaginal microbiome data, and market-basket data), with code publicly available.

Significance. If the proof gaps are fixed, the paper is a useful contribution: it provides the first consistency result for the core tensor in Tucker-based tensor topic modeling, with concrete entry-wise ℓ1 rates that depend only logarithmically on the vocabulary size for the topic matrix. The recovery identity G = S · (Ṽ(1), Ṽ(2), Ṽ(3)) is derived from the model rather than assumed, and the paper ships reproducible code and extensive experiments comparing against LDA, STM, Tensor-LDA, and NTD. The sparse extension in Assumption 2.4 is a sensible practical adaptation to Zipfian word frequencies. However, the theoretical guarantees as written rest on a literally circular proof of Lemma D.8 and on an unstated efficiency assumption for the vertex-hunting step, so the headline consistency claims are not fully supported in the submitted version.

major comments (2)

[Appendix D.2, Lemma D.8] The proof block of Lemma D.8 ends with the sentence "Proof. The proof of this lemma follows directly from Lemma D.8," which is self-referential and invalid. Equations (49)–(51) bound the Frobenius norms of Z(a)Z(a)ᵀ − E[Z(a)Z(a)ᵀ], and Lemma D.8 is used in Lemma E.3 to control the mode-3 perturbation ∥Q̂(3) − Q(3)∥. That perturbation control feeds the row-wise SCORE bound (E3), the vertex-hunting analysis in Lemma F.2, the A(3) rate in Lemma H.2, and finally the core-tensor rate in Lemma I.4 and Theorem 2.4. The preceding display does sketch a route through Lemmas D.6 and D.7, so the gap appears repairable, but as submitted the proof is not valid and the rates for A(3) and G in Theorem 2.4 are unsupported.
[Section 2.4 / Appendix F, Assumption F.1] Theorem 2.4 is stated under Assumptions 2.1, 2.2, and 2.3 only, but its proof via Lemmas H.1, H.2, and I.4 relies on Corollary F.0.1, which in turn invokes Assumption F.1: the vertex-hunting algorithm must satisfy max_k ∥ν̂_{π(k)} − ν_k∥ ≤ C max_i ∥ŷ_i − y_i∥. No proof is given that the successive projection algorithm (or any named algorithm) satisfies Assumption F.1 under the paper's assumptions; the text only cites prior work. Thus the theorem as stated omits a load-bearing condition. Either Assumption F.1 should be incorporated into Theorem 2.4, or the paper should verify the condition for a specific vertex-hunting algorithm under Assumptions 2.1–2.3.

minor comments (4)

[Appendix E.4] The phrase "sintheta theorem" should be replaced with a proper reference to Wedin's sin Θ theorem.
[Appendix M.1] The section heading "Arvix's Paper" is a typo; it should read "arXiv paper".
[Appendix C, Lemma C.3] The sentence "With these notations, Qi,M is the identity permutation of Qi,π" is confusing; the definition of the permutation-averaged statistic should be rewritten for clarity.
[Section 1.2] The definition of mode-1 matricization is grammatically incomplete ("We define the mode-1 matricization of a tensor X ... matrix X(1) with entries ..."); it should be rewritten as a complete sentence.

Circularity Check

1 steps flagged · score 4.0 of 10

One self-referential proof tag in Lemma D.8; the HOSVD/TTM core-tensor derivation is otherwise model-anchored and not circular.

other [Appendix D.2, Lemma D.8 (Frobenius concentration bounds (49)-(51)), terminal 'Proof.' line]
"Proof. The proof of this lemma follows directly from Lemma D.8"

Lemma D.8 is the lemma being proved. Its final proof line invokes Lemma D.8 itself, so the stated Frobenius-norm bounds (49)-(51) are not derived within the proof as printed; the conclusion is used to prove the conclusion. This is load-bearing for the mode-3 chain: Lemma E.3 controls ∥Z(3)Z(3)^⊤ - E[Z(3)Z(3)^⊤]∥ using the D.8 bound, which yields the mode-3 SVD perturbation (57), then the SCORE normalization error, the A(3) rate (18), and finally the core-tensor rate (19) in Theorem 2.4. The earlier displays in the same lemma sketch a derivation from Lemmas D.6 and D.7, so the circular tag is likely an erroneous duplicate line and the bound is plausibly recoverable; nevertheless, as submitted the lemma's proof is self-referential.

full rationale

The central derivation is not circular in the substantive sense: the Tucker model D = G·(A(1),A(2),A(3)) with multinomial noise is the input, and the estimator is constructed from the noisy tensor Y via HOSVD, SCORE normalization and vertex hunting, with the core recovered through the derived identity G = S·(V(1),V(2),V(3)) (Lemma I.1). No parameter is fitted to the target rates and then renamed a prediction; the constants in Theorem 2.4 are absolute. The paper's reliance on Ke and Wang [28] and on the authors' own [52] supplies building blocks (vertex hunting bounds, sparsity assumptions, threshold constants), but Theorem 2.4's A(1), A(2), A(3), and G rates are proven in-appendix from the multinomial noise lemmas rather than imported as the conclusion. The one genuine circular step is the duplicate last line of Lemma D.8, which proves Lemma D.8 from Lemma D.8; because that lemma feeds the mode-3 perturbation bound used in the core-tensor rate, the circular proof tag is a real defect, though the surrounding text indicates the bound was intended to follow from Lemmas D.6-D.7. The paper also candidly flags its strong anchor assumptions (Remark accompanying Assumption 2.2), which is a limitation rather than a circularity. Overall, the headline consistency claim has independent content and is not forced by self-definition, so the score reflects one localized self-referential proof rather than a reduction of the central result to its inputs.

Assumptions & free parameters 2 free parameters · 7 assumptions · 0 invented entities

The method depends on the stated nonnegative Tucker model and a cascade of identifiability, well-conditioning, and frequency assumptions. No new physical or mathematical entities are postulated; the core tensor is part of the Tucker model. The free parameters are the user-chosen ranks and the threshold constant, not quantities fitted to make the theory hold.

free parameters (2)

Tucker ranks K(1), K(2), K(3) = chosen by scree plots; examples: (2,3,4) for arXiv, (3,4,5) for microbiome, (4,4,10) for market basket
The number of latent clusters and topics is a model choice in the method. The theoretical analysis treats K(a) as fixed constants.
Low-frequency word threshold c' = 0.005 (as suggested in [52])
Equation (20) defines the surviving word set J using a threshold c' sqrt(log(NR)/(N(1)N(2)M)); the constant c' is chosen by hand, borrowing from the authors' prior work.

assumptions (7)

domain assumption Each document (i,j) follows a multinomial distribution with length Mij and expectation Dij* (Equation (2)).
This is the data generating model for tensor pLSI, stated in Section 1 and used throughout the proofs.
domain assumption The expected tensor D admits a nonnegative Tucker decomposition D = G · (A(1), A(2), A(3)) with row-wise stochastic A(1), A(2), column-wise stochastic A(3), and core slices summing to 1 (Equations (4)-(6)).
The low-rank Tucker structure with probabilistic normalization is the core modeling assumption, introduced in Section 1.
domain assumption Anchor conditions: every cluster in A(1) and A(2) has an anchor row, and every topic in A(3) has an anchor word (Assumption 2.2).
These conditions imply ideal simplex structure for the singular vector point clouds, enabling vertex hunting. The authors note this is stronger than in matrix topic models.
domain assumption Well-conditioning: the factor matrices and core matricizations have bounded condition numbers (Assumption 2.1).
Assumption 2.1 is stated as standard in the topic modeling literature [28, 31, 52].
domain assumption Topical correlation: all entries of A(3)^T A(3) are lower bounded by a constant (Assumption 2.3).
Assumption 2.3 ensures a spectral gap for the third mode and is used in Lemma E.5.
domain assumption Minimum word frequency: min_r f_r >= c sqrt(log(NR)/(N(1)N(2)M)) for the main theorem.
Stated in Theorem 2.4; needed to control the SCORE normalization and noise terms. The authors soften it with the sparsity assumption.
ad hoc to paper The chosen vertex hunting algorithm is efficient in the sense of Assumption F.1, with vertex error bounded by the point cloud error.
This is a technical assumption on the algorithm (SP, SVS, or AA) that the proofs rely on; it is defined specifically for this paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Tensor Topic Modeling Via HOSVD." pith.science (2026). https://pith.science/paper/WCP5HIDZ

@misc{pith2026250100535,
  author       = {Pith},
  title        = {Pith review of: Tensor Topic Modeling Via HOSVD},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WCP5HIDZ}},
  note         = {Machine review of arXiv:2501.00535}
}

abstract

By representing documents as mixtures of topics, topic modeling has allowed the successful analysis of datasets across a wide spectrum of applications ranging from ecology to genetics. An important body of recent work has demonstrated the computational and statistical efficiency of probabilistic Latent Semantic Indexing (pLSI)-- a type of topic modeling -- in estimating both the topic matrix (corresponding to distributions over word frequencies), and the topic assignment matrix. However, these methods are not easily extendable to the incorporation of additional temporal, spatial, or document-specific information, thereby potentially neglecting useful information in the analysis of spatial or longitudinal datasets that can be represented as tensors. Consequently, in this paper, we propose using a modified higher-order singular value decomposition (HOSVD) to estimate topic models based on a Tucker decomposition, thus accommodating the complexity of tensor data. Our method exploits the strength of tensor decomposition in reducing data to lower-dimensional spaces and successfully recovers lower-rank topic and cluster structures, as well as a core tensor that highlights interactions among latent factors. We further characterize explicitly the convergence rate of our method in entry-wise $\ell_1$ norm. Experiments on synthetic data demonstrate the statistical efficiency of our method and its ability to better capture patterns across multiple dimensions. Additionally, our approach also performs well when applied to large datasets of research abstracts and in the analysis of vaginal microbiome data.

Figures

Figures reproduced from arXiv: 2501.00535 by the authors.

**Figure 1.** Tucker components A(1) , A(2) , A(3) , G (Left to Right respectively) estimated by our method with ranks (2, 2, 3) for the tensor from [PITH_FULL_IMAGE:figures/full_fig_p018_1.png] view at source ↗

**Figure 2.** Tucker components A(1) , A(2) , A(3) , G (Left to Right respectively) estimated by Nonnegative Tucker decomposition (NTD) with ranks (2, 2, 3). The reconstruction error is ∥D − D∥ ˆ 1 = 91.543. variance in reconstruction error, as the estimation of the matrix W(3) is critical for determining A(1) , A(2). As shown in Appendix L.1, if the estimation of W(3) is sufficiently accurate, the rest of the procedure to estim… view at source ↗

**Figure 5.** Boxplot of the reconstruction errors over 30 runs on the same dataset from Figure [PITH_FULL_IMAGE:figures/full_fig_p018_5.png] view at source ↗

Figures from the paper (33 more)

**Figure 3.** Figure 3: Tucker components A(1) , A(2) , A(3) , G (Left to Right respectively) estimated by Tensor LDA with ranks (2, 2, 3). The reconstruction error is ∥D − D∥ ˆ 1 = 27.262 [PITH_FULL_IMAGE:figures/full_fig_p019_3.png]

**Figure 4.** Figure 4: Tucker components A(1) , A(2) , A(3) , G (Left to Right respectively) estimated by HybridLDA with ranks (2, 2, 3). The reconstruction error is ∥D − D∥ ˆ 1 = 82.703. 3.2 Varying the data dimensions We now vary the data dimensions and assess the performance of our metho…

**Figure 6.** Figure 6: TTM’s performance as a function of N(1), compared to various benchmarks. M is here set to 10,000 and K(1) = K(2) = 2 while K(3) = 4 [PITH_FULL_IMAGE:figures/full_fig_p020_6.png]

**Figure 7.** Figure 7: TTM’s performance as a function of R, compared to various benchmarks. M is here set to 10,000 and K(1), K(2) = 2 while K(3) = 4. We also evaluate the effect of the number of words per document M in [PITH_FULL_IMAGE:figures/full_fig_p020_7.png]

**Figure 8.** Figure 8: TTM’s performance as a function of M, compared to various benchmarks. R is here set to 500 and K(1) = K(2) = 2 while K(3) = 4. 4 Real data experiments In this section, we apply our method to real-world datasets, specifically focusing on text, microbiome as well as mark…

**Figure 9.** Figure 9: The Tucker components A(3) (right) and the core tensor (left) are derived from our method (TTM-HOSVD). The values in the core tensor have been sqrt-transformed in the colorbar to enhance visual interpretation. Each dot represents the frequency of the corresponding word…

**Figure 10.** Figure 10: Tucker components A(3) derived from benchmarks. It is required to permute the topics for comparison. Hybrid-LDA: v1: Statistics, v2: Physics, v3: Computer Science, v4: Math; STM: v1: Physics, v2: Math, v3: Computer Science, v4: Statistics; Tensor-LDA: v1: Physics, v2:…

**Figure 11.** Figure 11: Visualization of the construction of the vaginal microbiome tensor (24 [PITH_FULL_IMAGE:figures/full_fig_p023_11.png]

**Figure 12.** Figure 12: Comparison of the refinement and coherence of topics recovered using our method (a), [PITH_FULL_IMAGE:figures/full_fig_p025_12.png]

**Figure 14.** Figure 14: Dirichlet regression estimated coefficients (x-axis) quantifying the associations between [PITH_FULL_IMAGE:figures/full_fig_p025_14.png]

**Figure 13.** Figure 13: Left: Median Topic Resolution as a function of [PITH_FULL_IMAGE:figures/full_fig_p026_13.png]

**Figure 15.** Figure 15: Matrix components A(1) and A(2) derived from HOSVD. The y-axis of A(1) labels each subject’s racial and age information, with “UAB0XX” denoting the subject ID and the following number indicating the menstrual cycle number. The y-axis of A(2) corresponds to the four me…

**Figure 16.** Figure 16: Left: Core tensor derived from our method (TTM-HOSVD). Right: Some selected the [PITH_FULL_IMAGE:figures/full_fig_p027_16.png]

**Figure 17.** Figure 17: Topics recovered using our HO-SVD tensor topic modeling approach. Each dot repre [PITH_FULL_IMAGE:figures/full_fig_p029_17.png]

**Figure 18.** Figure 18: Estimated Latent Clusters along the time dimension. [PITH_FULL_IMAGE:figures/full_fig_p030_18.png]

**Figure 19.** Figure 19: Plate Diagram for the tensor LDA model In practice, we set the hyperparameters β, α1, α2 and α3 to 1. To fit this model, we use the variational Bayes provided in RStan[20]. This provides a fast and scalable solution to the model specified above. Throughout the paper, …

**Figure 20.** Figure 20: Plate Model for the LDA model fitted in Step 1 of the Hybrid LDA approach [PITH_FULL_IMAGE:figures/full_fig_p078_20.png]

**Figure 21.** Figure 21: Nonnegative 30 × 10 × 50 (ground-truth) tensor D where the slices are taken along the second mode. L.1 Tucker decomposition: Illustration and Comparison In this section, we provide a detailed discussion and illustration of why Tucker decomposition is our preferred app…

**Figure 22.** Figure 22: Tucker components A(1) , A(2) , A(3) , G (Left to Right respectively) estimated by our method with ranks (2, 2, 3) for the tensor from [PITH_FULL_IMAGE:figures/full_fig_p080_22.png]

**Figure 23.** Figure 23: NNCPD components A(1) , A(2) , A(3) (Left to right, respectively) for both subplots (a) and (b). Components A(1) , A(2), and A(3) representing the factor matrices corresponding to the first (document), second (time), and third (word) modes, respectively. Plot (a) and …

**Figure 24.** Figure 24: The Matrix LDA components A(3) and W(3)⊤ are shown from left to right in both subplots (a) and (b). Here, A(3) represents the word-topic matrix, while W(3) denotes the topic proportions for each document at different time points. Plot (a) illustrates the results from …

**Figure 25.** Figure 25: The post-processing Matrix LDA components [PITH_FULL_IMAGE:figures/full_fig_p081_25.png]

**Figure 26.** Figure 26: The Box plot of the reconstruction errors over 30 repeated runs on the same dataset [PITH_FULL_IMAGE:figures/full_fig_p081_26.png]

**Figure 27.** Figure 27: Scree plot of D(a) from TTM-HOSVD 83 [PITH_FULL_IMAGE:figures/full_fig_p083_27.png]

**Figure 28.** Figure 28: Core tensor components G derived from different methods. Time Slice [,1] [,2] [,3] Statistics Theoretical Group 0.11 0.08 0.16 Application Group 0.08 0.06 0.08 Total 0.19 0.14 0.24 Physics Theoretical Group 0.14 0.16 0.15 Application Group 0.00 0.00 0.00 Total 0.14 0.…

**Figure 29.** Figure 29: Topic refinement and resolution as a function of [PITH_FULL_IMAGE:figures/full_fig_p085_29.png]

**Figure 30.** Figure 30: Dirichlet regression estimated coefficients (x-axis) quantifying the associations between [PITH_FULL_IMAGE:figures/full_fig_p087_30.png]

**Figure 31.** Figure 31: Matrix component A(2) derived from all methods. The color scale has been sqrttransformed. Remind that we select 3 days from the follicular phase, 2 days from the menstrual phase, 4 days from the luteal phase, and 2 days from ovulation. The columns in the plots need t…

**Figure 32.** Figure 32: The topic proportions W(3) by different methods throughout the menstrual cycle (a) TTM-HOSVD (b) Hybrid-LDA (c) Tensor-LDA (d) NTD [PITH_FULL_IMAGE:figures/full_fig_p089_32.png]

**Figure 33.** Figure 33: Core tensor obtained by different methods. Groups of subject align with the Race [PITH_FULL_IMAGE:figures/full_fig_p089_33.png]

**Figure 34.** Figure 34: Market-basket topics recovered by STM. 90 [PITH_FULL_IMAGE:figures/full_fig_p090_34.png]

**Figure 35.** Figure 35: Topics recovered using Hybrid-LDA. 91 [PITH_FULL_IMAGE:figures/full_fig_p091_35.png]

**Figure 36.** Figure 36: Topics recovered using Topic-Score HOSVD. [PITH_FULL_IMAGE:figures/full_fig_p092_36.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

101 extracted references · 76 canonical work pages

[1]

Agarwal and B.-C

D. Agarwal and B.-C. Chen. flda: matrix factorization through latent dirichlet allocation. In Proceedings of the third ACM international conference on Web search and data mining , pages 91–100, 2010

2010
[2]

M. Ahn, N. Eikmeier, J. Haddock, L. Kassab, A. Kryshchenko, K. Leonard, D. Needell, R. Madushani, E. Sizikova, and C. Wang. On large-scale dynamic topic modeling with non- negative cp tensor decomposition. Advances in Data Science , pages 181–210, 2021

2021
[3]

Anandkumar, R

A. Anandkumar, R. Ge, D. J. Hsu, S. M. Kakade, M. Telgarsky, et al. Tensor decompositions for learning latent variable models. J. Mach. Learn. Res. , 15(1):2773–2832, 2014

2014
[4]

M. C. U. Ara´ ujo, T. C. B. Saldanha, R. K. H. Galvao, T. Yoneyama, H. C. Chame, and V. Visani. The successive projections algorithm for variable selection in spectroscopic multi- component analysis. Chemometrics and Intelligent Laboratory Systems , 57(2):65–73, 2001

2001
[5]

Arora, R

S. Arora, R. Ge, and A. Moitra. Learning topic models–going beyond svd. In 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science , pages 1–10. IEEE, 2012

2012
[6]

Bhaskara, M

A. Bhaskara, M. Charikar, and A. Vijayaraghavan. Uniqueness of tensor decompositions with applications to polynomial identifiability. In Conference on Learning Theory , pages 742–778. PMLR, 2014

2014
[7]

X. Bing, F. Bunea, and M. Wegkamp. A fast algorithm with minimax optimal guarantees for topic models with an unknown number of topics. Bernoulli, 2020

2020
[8]

X. Bing, F. Bunea, and M. Wegkamp. Optimal estimation of sparse topic models. The Journal of Machine Learning Research, 21(1):7189–7233, 2020

2020

Show all 101 references

[9]

D. M. Blei, A. Y. Ng, and M. I. Jordan. Latent dirichlet allocation. Journal of Machine Learning Research, 3(Jan):993–1022, 2003

2003
[10]

T. T. Cai and H. H. Zhou. Optimal rates of convergence for sparse covariance matrix estima- tion. The Annals of Statistics , 2012

2012
[11]

Cha and J

Y. Cha and J. Cho. Social-network analysis using topic models. In Proceedings of the 35th international ACM SIGIR conference on Research and development in information retrieval , pages 565–574, 2012

2012
[12]

X. Chen, T. He, X. Hu, Y. Zhou, Y. An, and X. Wu. Estimating functional groups in human gut microbiome with probabilistic topic models. IEEE transactions on nanobioscience , 11(3):203– 215, 2012

2012
[13]

Y. Chen, S. He, Y. Yang, and F. Liang. Learning topic models: Identifiability and finite-sample analysis. Journal of the American Statistical Association , 118(544):2860–2875, 2023

2023
[14]

Cichocki, R

A. Cichocki, R. Zdunek, and S.-i. Amari. Nonnegative matrix and tensor factorization [lecture notes]. IEEE signal processing magazine , 25(1):142–145, 2007

2007
[15]

S. A. Curiskis, B. Drake, T. R. Osborn, and P. J. Kennedy. An evaluation of document clustering and topic modelling in two online social networks: Twitter and reddit. Information Processing & Management, 57(2):102034, 2020. 33

2020
[16]

Cutler and L

A. Cutler and L. Breiman. Archetypal analysis. Technometrics, 36(4):338–347, 1994

1994
[17]

De Lathauwer, B

L. De Lathauwer, B. De Moor, and J. Vandewalle. On the best rank-1 and rank-(r 1, r 2,..., rn) approximation of higher-order tensors. SIAM journal on Matrix Analysis and Applications , 21(4):1324–1342, 2000

2000
[18]

Dethlefsen and D

L. Dethlefsen and D. A. Relman. Incomplete recovery and individualized responses of the human distal gut microbiota to repeated antibiotic perturbation. Proceedings of the National Academy of Sciences, 108(supplement 1):4554–4561, 2011

2011
[19]

Fukuyama, K

J. Fukuyama, K. Sankaran, and L. Symul. Multiscale analysis of count data through topic alignment. arXiv preprint arXiv:2109.05541 , 2021

2021 arXiv
[20]

J. Guo, J. Gabry, B. Goodrich, and S. Weber. Package ’rstan’.https://pj.freefaculty.org/ guides/crmda_workshops/sem/Archive/sem-4/literature/manuals/rstan.pdf, 2020

2020
[21]

X. Guo, Y. Xiang, Q. Chen, Z. Huang, and Y. Hao. Lda-based online topic detection using tensor factorization. Journal of information science , 39(4):459–469, 2013

2013
[22]

T. Hofmann. Probabilistic latent semantic indexing. In Proceedings of the 22nd annual in- ternational ACM SIGIR Conference on Research and Development in Information Retrieval , pages 50–57, 1999

1999
[23]

Hood and A

J. Hood and A. Schein. The all0core tensor decomposition for sparse count data. arXiv preprint arXiv:2403.06153, 2024

2024 arXiv
[24]

R. A. Horn and C. R. Johnson. Matrix analysis . Cambridge University Press, 2012

2012
[25]

Javadi and A

H. Javadi and A. Montanari. Nonnegative matrix factorization via archetypal analysis. Journal of the American Statistical Association , 115(530):896–907, 2020

2020
[26]

J. Jin. Fast community detection by score. The Annals of Statistics , 2015

2015
[27]

J. Jin, Z. T. Ke, and S. Luo. Estimating network memberships by simplex vertex hunting. arXiv preprint arXiv:1708.07852 , 12, 2017

2017 arXiv
[28]

Z. T. Ke and M. Wang. Using svd for topic modeling. Journal of the American Statistical Association, pages 1–16, 2022

2022
[29]

S. J. Kho, H. B. Yalamanchili, M. L. Raymer, and A. P. Sheth. A novel approach for classifying gene expression data using topic modeling. In Proceedings of the 8th ACM international conference on bioinformatics, computational biology, and health informatics , pages 388–393, 2017

2017
[30]

Kim and S

Y.-D. Kim and S. Choi. Nonnegative tucker decomposition. In 2007 IEEE conference on computer vision and pattern recognition , pages 1–8. IEEE, 2007

2007
[31]

Klopp, M

O. Klopp, M. Panov, S. Sigalla, and A. Tsybakov. Assigning topics to documents by successive projections. arXiv preprint arXiv:2107.03684 , 2021

2021 arXiv
[32]

H. Lee, J. Kihm, J. Choo, J. Stasko, and H. Park. ivisclustering: An interactive visual document clustering via topic modeling. In Computer graphics forum, volume 31, pages 1155–
[33]

Lei and A

J. Lei and A. Rinaldo. Consistency of spectral clustering in stochastic block models. The Annals of Statistics , pages 215–237, 2015

2015
[34]

L.-J. Li, C. Wang, Y. Lim, D. M. Blei, and L. Fei-Fei. Building and using a semantivisual image hierarchy. In 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pages 3336–3343. IEEE, 2010

2010
[35]

L. Liu, L. Tang, W. Dong, S. Yao, and W. Zhou. An overview of topic modeling and its current applications in bioinformatics. SpringerPlus, 5:1–22, 2016

2016
[36]

Z. Ma. Sparse principal component analysis and iterative thresholding. The Annals of Statis- tics, 2013

2013
[37]

Manzonelli, W

N. Manzonelli, W. Zhang, and S. Vadhan. Membership inference attacks and privacy in topic modeling. arXiv preprint arXiv:2403.04451 , 2024

2024 arXiv
[38]

Mørup and L

M. Mørup and L. K. Hansen. Archetypal analysis for machine learning and data mining. Neurocomputing, 80:54–63, 2012

2012
[39]

Movassagh, L

M. Movassagh, L. M. Bebell, K. Burgoine, C. Hehnly, L. Zhang, K. Moran, K. Sheldon, S. A. Sinnar, E. Mbabazi-Kabachelor, E. Kumbakumba, et al. Vaginal microbiome topic modeling of laboring ugandan women with and without fever. npj Biofilms and Microbiomes , 7(1):75, 2021

2021
[40]

T. Okui. A bayesian nonparametric topic model for microbiome data using subject attributes. IPSJ Transactions on Bioinformatics , 13:1–6, 2020

2020
[41]

Paatero and U

P. Paatero and U. Tapper. Positive matrix factorization: A non-negative factor model with optimal utilization of error estimates of data values. Environmetrics, 5(2):111–126, 1994

1994
[42]

J. K. Pritchard, M. Stephens, and P. Donnelly. Inference of population structure using multi- locus genotype data. Genetics, 155(2):945–959, 2000

2000
[43]

Richard and A

E. Richard and A. Montanari. A statistical model for tensor pca. Advances in neural infor- mation processing systems, 27, 2014

2014
[44]

M. E. Roberts, B. M. Stewart, D. Tingley, C. Lucas, J. Leder-Luis, S. K. Gadarian, B. Al- bertson, and D. G. Rand. Structural topic models for open-ended survey responses. American journal of political science , 58(4):1064–1082, 2014

2014
[45]

Romeo, S

S. Romeo, S. Greco, and A. Tagarelli. Multi-topic and multilingual document clustering via tensor modeling. PhD thesis, University of Calabria Arcavacata di Rende,, 2014

2014
[46]

Romero, S

R. Romero, S. S. Hassan, P. Gajer, A. L. Tarca, D. W. Fadrosh, L. Nikita, M. Galuppi, R. F. Lamont, P. Chaemsaithong, J. Miranda, et al. The composition and stability of the vaginal microbiota of normal pregnant women is different from that of non-pregnant women. Microbiome, 2...

2014
[47]

Sankaran and S

K. Sankaran and S. P. Holmes. Latent variable modeling for the microbiome. Biostatistics, 20(4):599–614, 2019

2019
[48]

K. Schacke. On the kronecker product. Master’s thesis, University of Waterloo , 2004. 35

2004
[49]

Shalon, R

D. Shalon, R. N. Culver, J. A. Grembi, J. Folz, P. V. Treit, H. Shi, F. A. Rosenberger, L. Dethlefsen, X. Meng, E. Yaffe, et al. Profiling the human intestinal environment under physiological conditions. Nature, 617(7961):581–591, 2023

2023
[50]

Sharma, A

V. Sharma, A. Kumar, N. Agrawal, P. Singh, and R. Kulshreshtha. Image summarization using topic modelling. In 2015 IEEE International Conference on Signal and Image Processing Applications (ICSIPA), pages 226–231. IEEE, 2015

2015
[51]

Symul, P

L. Symul, P. Jeganathan, E. K. Costello, M. France, S. M. Bloom, D. S. Kwon, J. Ravel, D. A. Relman, and S. Holmes. Sub-communities of the vaginal microbiota in pregnant and non-pregnant women. Proceedings of the Royal Society B , 290(2011):20231461, 2023

2011
[52]

H. Tran, Y. Liu, and C. Donnat. Sparse topic modeling via spectral decomposition and thresholding. arXiv preprint arXiv:2310.06730 , 2023

2023 arXiv
[53]

J. A. Tropp et al. An introduction to matrix concentration inequalities. Foundations and Trends® in Machine Learning , 8(1-2):1–230, 2015

2015
[54]

N. A. Tu, D.-L. Dinh, M. K. Rasel, and Y.-K. Lee. Topic modeling and improvement of image representation for large-scale image retrieval. Information Sciences, 366:99–120, 2016

2016
[55]

Vershynin

R. Vershynin. High-dimensional probability: An introduction with applications in data science , volume 47. Cambridge university press, 2018

2018
[56]

P.- ˚A. Wedin. Perturbation bounds in connection with singular value decomposition. BIT Numerical Mathematics, 12:99–111, 1972

1972
[57]

Wu and M

Y. Wu and M. Ester. Flame: A probabilistic model combining aspect based opinion mining and collaborative filtering. In Proceedings of the eighth ACM international conference on web search and data mining , pages 199–208, 2015

2015
[58]

Xia and F

D. Xia and F. Zhou. The sup-norm perturbation of hosvd and low rank tensor denoising. Journal of Machine Learning Research , 20(61):1–42, 2019

2019
[59]

Xie and E

P. Xie and E. P. Xing. Integrating document clustering and topic modeling. arXiv preprint arXiv:1309.6874, 2013

2013 arXiv
[60]

J. Yang, X. Feng, A. F. Laine, and E. D. Angelini. Characterizing alzheimer’s disease with image and genetic biomarkers using supervised topic models. IEEE journal of biomedical and health informatics, 24(4):1180–1187, 2019

2019
[61]

C.-K. Yau, A. Porter, N. Newman, and A. Suominen. Clustering scientific documents with topic modeling. Scientometrics, 100:767–786, 2014

2014
[62]

Zhang and R

A. Zhang and R. Han. Optimal sparse singular value decomposition for high-dimensional high-order data. Journal of the American Statistical Association , 2019

2019
[63]

Zhang and D

A. Zhang and D. Xia. Tensor svd: Statistical and computational limits. IEEE Transactions on Information Theory , 64(11):7311–7338, 2018

2018
[64]

Zheng, W

X. Zheng, W. Ding, Z. Lin, and C. Chen. Topic tensor factorization for recommender system. Information Sciences, 372:276–293, 2016. 36

2016
[65]

Y. Zhou, J. Fan, and L. Xue. How much can machines learn finance from chinese text data? Management Science, 2024

2024
[66]

G. K. Zipf. The Psycho-biology of Language: An Introduction to Dynamic Philology. Houghton- Mifflin, 1936. 37 A Appendix: Proofs B Properties of the data B.1 Tensor Topic Modeling In this paper, we generalize the usual matrix topic modeling framework assumed by Probabilistic L...

1936
[68]

M1(D) = D(1) = A(1)W(1) ∈ RN (1)×(N (2)R), which we assume to have rank K(1),
[69]

M2(D) = D(2) = A(2)W(2) ∈ RN (2)×(N (1)R), which we assume to have rank K(2),
[70]

Given the inherent structure of the problem, each one of the Tucker factor matrices A(1), A(2), A(3) belong to one of two matrix types

M3(D) = D(3) = A(3)W(3) ∈ RR×(N (1)N (2)), which we assume to have rank K(3). Given the inherent structure of the problem, each one of the Tucker factor matrices A(1), A(2), A(3) belong to one of two matrix types. Letting A ∈ Rn×K denote a general matrix, the factor matrices c...
[71]

Expectation: In this case, given observation (31) above, the expectation 1 N (1) P i,r Z 2 ijr simplifies to: 1 N (1) X i,r E[Z 2 ijr ] = 1 N (1) X i,r 1 M Dijr (1 − Dijr ) = 1 M − 1 M N(1) X i,r D2 ijr
[72]

It is easy to see that | ˜Qijm| ⩽ 2

First summand: Define ˜Qijm = P r∈[R] X 2 ijm(r). It is easy to see that | ˜Qijm| ⩽ 2. Indeed, letting r0 denote the index for which Tijm(r0) = 1, we have: ˜Qijm = X r∈[R] Tijm(r) − 2Tijm(r)Dijr + D2 ijr ) ≤ 1 − 2Dijr0 + 1 ≤ 2. 49 Furthermore, we have: E[ ˜Qijm] = P r Dijr (1 ...
[73]

Second summand: For the terms involvingm ̸= m′, we define Qijmm′ = P r Xijm(r)Xijm′(r). In this case, it is also possible to verify that, for fix i, j, m, m′, |Qijmm′| ⩽ 2, E[Qijmm′] = 0, Var[Qijmm′] = X r,r′ E[Xijm(r)Xijm′(r)Xijm(r′)Xijm′(r′)] − E[Qijmm′]2 = X r,r′ E[Xijm(r)X...
[74]

First summand: we have, for any tuplei, j, r: |Xijm(r)2| ⩽ 2, E[Xijm(r)2] = Dijr −D 2 ijr and Var[ Xijm(r)2] ⩽ Dijr ⩽ fr (see Appendix B.2, noting that here documents are indexed by the tuple ( i, j)). By direct application of the Bernstein Inequality (Lemma C.1), as long asfr...
[75]

Second summand: For m ̸= m′, we consider Qr ijm = Xij(2m−1)(r)Xij(2m)(r). It is easy to see that for any fixed r, the Qrs are such that, for any tuple of indices ( i, j, m): |Qr ijm| = |Xij(2m−1)(r)Xij(2m)(r)| ⩽ 1, E[Qr ijm] = E[Xij(2m−1)(r)Xij(2m)(r)] = 0 and Var[Xij(2m−1)(r)...
[76]

Then: 1 N (2) N (2),RX j,r=1 Zijr Zi′jr = 1 M 2N (2) N (2) X j MX m,m′=1 Qjmm′ = 1 M 2N (2) N (2) X j MX m Qjmm + 1 M 2N (2) N (2) X j MX m̸=m′ Qjmm′

Adapting Equation (42), we have for all i ̸= i′: 1 N (2) N (2),RX j,r=1 Zijr Zi′jr = 1 M 2N (2) N (2),RX j,r=1 MX m,m′ Xijm(r)Xi′jm′(r) For a fixed tuple i, i′ such that i ̸= i′, let Qjmm′ = P r∈[R] Xijm(r)Xi′jm′(r). Then: 1 N (2) N (2),RX j,r=1 Zijr Zi′jr = 1 M 2N (2) N (2) X...
[77]

By direct application of Cauchy-Schwarz: |Qjmm| ≤ sX r Xijm(r)2 sX r Xijm′(r)2 ≤ sX r Xijm(r) sX r Xijm′(r) = 1. By independence of the variables Xijm(r) for i ̸= i′, we have: E[Qjmm] = 0, and Var[Qjmm] = X r1 X r2 Cov Xijm(r1)Xi′jm(r1), Xijm(r2)Xi′jm(r2) ⩽ X r1 X r2 Dijr1Dijr...
[78]

E[Qjmm′] = 0, Var[Qjmm′] ⩽ c

For m ̸= m′, Qjmm′ = P r Xijm(r)Xi′jm′(r) also verifies |Qjmm′| ⩽ 1. E[Qjmm′] = 0, Var[Qjmm′] ⩽ c. By application of Lemma C.3, we obtain that with probability at least 1 − o(N −1 R ): 1 N (2)M ! X j,π X m Qi′ij,π ⩽ C∗ r M log(NR) N (2) • Mode 3 matricization . We treat the ca...
[79]

By Lemma C.1, if fr ⩾ q log(NR) M N(1)N (2) , with high probability, 1 M N(1)N (2) X ijm Xijm(r)Xijm(r′) − E[⋆] ⩽ C∗ r fr′fr log(NR) M N(1)N (2)

|Xijm(r)Xijm(r′)| ⩽ 1, E[Xijm(r)Xijm(r′)] = −Dijr Dijr ′ and Var[ Xijm(r)Xijm(r′)] ⩽ frfr′. By Lemma C.1, if fr ⩾ q log(NR) M N(1)N (2) , with high probability, 1 M N(1)N (2) X ijm Xijm(r)Xijm(r′) − E[⋆] ⩽ C∗ r fr′fr log(NR) M N(1)N (2)
[80]

Let ˜M = ⌊M/2⌋

For m ̸= m′, consider Xij(2m−1)(r)Xij(2m)(r′) such that |Xij(2m−1)(r)Xij(2m)(r′)| ⩽ 1, and E[Xij(2m−1)(r)Xij(2m)(r′)] = 0 and Var[Xij(2m−1)(r)Xij(2m)(r′)] ⩽ Dijr Dijr ′ ⩽ frfr′. Let ˜M = ⌊M/2⌋. Applying Lemma C.3, P   1 N (1,2) 1 M ! X π∈Π[M ] X i,j ˜MX m=1 Xij(2m−1)(r)Xij(2...
[81]

The matrix ˜V = WBΛ−1 ∈ RK×K is invertible, unique and such that ˜V = (Ξ⊤A)−1
[82]

˜V ˜V⊤ = (A⊤A)−1, and thus the singular values of ˜V are the inverses of the singular values of A
[83]

˜V⋆K are right eigenvectors of WW⊤A⊤A associated with eigenvalues λ2 k(these eigen- vectors are not necessarily orthogonal with each other)

˜V⋆1, . . .˜V⋆K are right eigenvectors of WW⊤A⊤A associated with eigenvalues λ2 k(these eigen- vectors are not necessarily orthogonal with each other)
[84]

Finally, we have Ξ = A ˜V, and ∥Ξj⋆∥2 ⩽ ∥Aj⋆∥2σK(A)−1. Proof. The proof for the previous lemma is essentially a readaptation of Lemma D.2 in [28], which we provide here to make this manuscript self-contained, but we emphasize that the original proof in a more general setting i...
[85]

(31) Since: Ξ⊤Ξ = IK = Ξ⊤A ˜V = (Ξ⊤A) ˜V, we deduce that ˜V is unique and non singular, and that it is the inverse of Ξ⊤A

By definition: D = ΞΛB⊤, so we write: Ξ = DBΛ−1 = A(WBΛ−1) = A ˜V. (31) Since: Ξ⊤Ξ = IK = Ξ⊤A ˜V = (Ξ⊤A) ˜V, we deduce that ˜V is unique and non singular, and that it is the inverse of Ξ⊤A. 54
[86]

In this case, plugging in Ξ = A ˜V in Ξ⊤Ξ = IK, we obtain: ˜V⊤A⊤A ˜V = IK Multiplying left and right by ˜V and ˜V⊤ respectively: ( ˜V ˜V⊤)A⊤A( ˜V ˜V⊤) = ˜V ˜V⊤ =⇒ ˜V ˜V⊤ = (A⊤A)−1
[87]

Then, for any column ξk of the matrix Ξ, by definition: DDT ξk = λ2 kξk where by (31), ξk = A ˜V·k

Let λk denote the singular values of D = A W. Then, for any column ξk of the matrix Ξ, by definition: DDT ξk = λ2 kξk where by (31), ξk = A ˜V·k. Therefore: DDT ξk = A WWT W AT A ˜V·k = λ2 kA ˜V·k =⇒ AT A WWT W AT A ˜V·k = λ2 kAT A ˜V·k =⇒ WT W AT A ˜V·k = λ2 k ˜V·k where the ...
[88]

By (31), Ξ = A ˜V. The inequality follows by definition of the operator norm of ˜V, and by application of point 2 in which we noted that the singular values of ˜V are the inverses of the singular values of A. E.1 Properties of the SVD Proposition E.1 (Lemma 5.1 in [33]) . Let ...
[89]

Analysis of D (3)(Z(3))⊤ + Z(3)(D(3))⊤: We have: D(3)(Z(3))⊤ = X k A(3) ⋆k (Z(3)(W(3) k⋆ )⊤)⊤. Using Lemma D.9 and the fact that P k ∥A⋆k∥2 ⩽ P k ∥A⋆k∥1 = K(3) and P r fr ⩽ K(3) ∥D(3)(Z(3))⊤ + Z(3)(D(3))⊤∥F ⩽ 2∥D(3)(Z(3))⊤∥F ⩽ 2 X k ∥A(3) ⋆k ∥2∥Z(3)(W(3) k⋆ )⊤∥2 = 2 X k ∥A(3) ...
[90]

Analysis of ∥Z(3)(Z(3))⊤ −E[Z(3)(Z(3))⊤]∥op: We use the upper bound given by Lemma D.8: ∥Z(3)(Z(3))⊤ − E[Z(3)(Z(3))⊤]∥ ⩽ C∗ r N (1,2) log NR M !
[91]

Analysis of ∥E[Z(3)(Z(3))⊤] − L(3) + 1 M D(3)(D(3))⊤∥: We use Lemmas D.1 and D.5: ∥E[Z(3)(Z(3))⊤]−L(3)+ 1 M D(3)(D(3))⊤∥ = max r 1 M | X ij Zijr | ⩽ max r C∗ r frN (1,2) log NR M ! Using the fact that fr ⩽ K(3), we have ∥ ˆQ(3) − Q(3)∥ ⩽ r N (1,2) log NR M . Applying propositi...
[92]

We have ∥Ξ(1) i⋆ ∥2 2 ⩽ ∥A(1) i⋆ ∥2 2σK(A(1))−2 ⩽ ∥A(1) i⋆ ∥1σK(A(1))−2 ⩽ 1/N(1)

Let ˆU(1) = ˆΞ(1), U∗(1) = U(1) = Ξ(1). We have ∥Ξ(1) i⋆ ∥2 2 ⩽ ∥A(1) i⋆ ∥2 2σK(A(1))−2 ⩽ ∥A(1) i⋆ ∥1σK(A(1))−2 ⩽ 1/N(1)
[93]

The first group only includes the leading eigenvector Ξ(3) ⋆1 as s = 1, k= 1, while the second group contains the left K − 1 eigenvectors as s = 2 and k = K(3) in lemma E.2

For ∥( ˆQ(1) − Q(1))i⋆∥, we have ∥( ˆQ(1) − Q(1))i⋆∥ =∥Y(1)(Y(1) − D(1))i⋆ + (Y(1) − D(1))D(1) i⋆ ∥ ⩽∥Y(1)∥∥Z(1) i⋆ ∥ + ∥Z(1)∥∥D(1) i⋆ ∥ ⩽ p N (1,2)∥Z(1) i⋆ ∥ + r N (1,2) log NR M ∥D(1) i⋆ ∥ ⩽N (2) r N (1) log NR M 60 where the last second inequality is given by Wely’s inequal...
[94]

Let U∗(3) = Ξ(3), we have ∥Ξ(3) r⋆ ∥2 2 ⩽ ∥Ar⋆∥2 2σK(A)−2 ⩽ ∥Ar⋆∥1σK(A)−2 ⩽ fr/K
[95]

The upper bound ∥(Q(3) − ˆQ(3))r⋆∥2 ⩽ C∗ q frN (1)N (2) log(NR) M . w.h.p. is given by ∥(D(3)Z(3)⊤ + Z(3)D(3)⊤)r⋆∥2 ⩽ X k Ark∥Z(3)Wk⋆∥ + X k X ij Zijr Wk,ij ∥A⋆k∥2 Lemma D.9 ≲ r frN (1,2) log NR M Z(3)Z(3)⊤ − E[Z(3)Z(3)⊤] r⋆ ⩽ vuuut X r′  X ij Zijr Zijr ′ − E[Zijr Zijr ′]  ...
[96]

Therefore, we have the non-degeneracy of ˆV∗(a) by lemma G.1

ˆV∗(a) for a ∈ {1, 2} is non-degenerate with σmin( ˆV∗(a)) ⩾ 1 2 q c∗ N (a) : Since Corollary F.0.1 yields that Π(a) ˆV∗(a)O(a) − V∗(a) F ⩽ C∗ r log NR N (a)M for any a ∈ {1, 2}, 68 and Lemma F.3 gives σmin(V∗(a)) ⩾ q c∗ N (a) , we can use the assumption M ⩾ log NR/c0 with c0 ...
[97]

Lemma F.1 can further admit that ˆS∗(a)O − S∗(a) 2 F = P i (ˆS∗(a)O − S∗(a))i⋆ 2 2 ≲ log NR M . Then, (73) gives ˜Ω∗(a) − Ω∗(a)Π(a) F ≲ r log NR M p N (a) + N (a) · r log NR N (a)M ⩽ r N (a) log NR M for a ∈ {1, 2} • Mode 3 matricization: with probability at least 1 − o(N −1 R ),
[98]

By the assumption fr ⩾ q log NR N (1,2)M , Corollary F.0.1 gives that Π(3) ˆV∗(3)O(3) − V∗(3) F ≲ log NR N (1,2)M 1/4

ˆV∗(3) is non-degenerate with σmin( ˆV∗(3)) ⩾ c∗/2: Lemma F.4 gives σmin(V∗(3)) ⩾ c∗. By the assumption fr ⩾ q log NR N (1,2)M , Corollary F.0.1 gives that Π(3) ˆV∗(3)O(3) − V∗(3) F ≲ log NR N (1,2)M 1/4 . As c∗ − C∗ log NR N (1,2)M 1/4 ⩾ c∗ − o(1) ⩾ c∗/2 > 0, Lemma G.1 gives ...
[99]

with O(3) = diag(±1, Ω∗)

As S∗(3) = [1R, S(3)], and ˆS∗(3) = [1R, ˆS(3)], Lemma F.2 also yields that (ˆS∗(3)O(3) − S∗(3))r⋆ ≲ q log NR frN (1,2)M . with O(3) = diag(±1, Ω∗). By (72), we have ∥S∗(3) r⋆ ∥2 ⩽ q 1 + ∥S(3) r⋆ ∥2 2 ⩽ C∗. Then, (74) gives ∥ ˜Ω∗(3) r⋆ − Π(3)⊤Ω∗(3) r⋆ ∥2 ≲ s log NR frN (1,2)M ...
[100]

Matrix decomposition is inherently limited to two modes (e.g., customers at a given time and products), which fails to preserve the multidimensional relationships in the data

Increasing the rank to 4, as shown in Figure 23b, does not resolve this limitation, underscoring that higher ranks alone cannot fully capture such nuanced changes, and highlighting the need for more sophisticated decompositions that incorporate additional interactions. Matrix ...

2007
[101]

canned tomatoes 16 oz

given from STM. Vertical error bars represent the interquartile range for the average topic resolution scores over 25 trials. Additional results for (ii): Compared to the post-processing matrix methods, such as STM and Hybrid-LDA, the Tucker-based decomposition methods (as sho...
[1164]

Wiley Online Library, 2012. 34

2012

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [1]

Agarwal and B.-C

D. Agarwal and B.-C. Chen. flda: matrix factorization through latent dirichlet allocation. In Proceedings of the third ACM international conference on Web search and data mining , pages 91–100, 2010

2010

[2] [2]

M. Ahn, N. Eikmeier, J. Haddock, L. Kassab, A. Kryshchenko, K. Leonard, D. Needell, R. Madushani, E. Sizikova, and C. Wang. On large-scale dynamic topic modeling with non- negative cp tensor decomposition. Advances in Data Science , pages 181–210, 2021

2021

[3] [3]

Anandkumar, R

A. Anandkumar, R. Ge, D. J. Hsu, S. M. Kakade, M. Telgarsky, et al. Tensor decompositions for learning latent variable models. J. Mach. Learn. Res. , 15(1):2773–2832, 2014

2014

[4] [4]

M. C. U. Ara´ ujo, T. C. B. Saldanha, R. K. H. Galvao, T. Yoneyama, H. C. Chame, and V. Visani. The successive projections algorithm for variable selection in spectroscopic multi- component analysis. Chemometrics and Intelligent Laboratory Systems , 57(2):65–73, 2001

2001

[5] [5]

Arora, R

S. Arora, R. Ge, and A. Moitra. Learning topic models–going beyond svd. In 2012 IEEE 53rd Annual Symposium on Foundations of Computer Science , pages 1–10. IEEE, 2012

2012

[6] [6]

Bhaskara, M

A. Bhaskara, M. Charikar, and A. Vijayaraghavan. Uniqueness of tensor decompositions with applications to polynomial identifiability. In Conference on Learning Theory , pages 742–778. PMLR, 2014

2014

[7] [7]

X. Bing, F. Bunea, and M. Wegkamp. A fast algorithm with minimax optimal guarantees for topic models with an unknown number of topics. Bernoulli, 2020

2020

[8] [8]

X. Bing, F. Bunea, and M. Wegkamp. Optimal estimation of sparse topic models. The Journal of Machine Learning Research, 21(1):7189–7233, 2020

2020