Pith. sign in

REVIEW 2 major objections 5 minor 46 references

Learning document embeddings along with their uncertainties

T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that representing each document as a Gaussian distribution, not a single point vector, yields better perplexity and topic-identification accuracy than point-estimate models.

desk verdict Solid Bayesian SMM extension with a useful uncertainty-aware classifier; the perplexity claim is undermined by an apples-to-oranges ELBO comparison but the paper deserves review. read the letter →

arxiv 1908.07599 v3 pith:VGBHSI6Z submitted 2019-08-20 cs.CL cs.LG

Santosh Kesiraju , Oldřich Plchot , Lukáš Burget , Suryakanth V Gangashetty This is my paper · ORCID
classification cs.CLcs.LG
keywords BayesiansubspacemultinomialmodeldocumentembeddingsuncertaintyvariationalinferencetopicidentificationperplexityGaussianlinearclassifierreparameterizationtrick
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a document is better represented by a Gaussian distribution over embeddings than by a single point vector, both for fitting text and for identifying its topic. It proposes the Bayesian subspace multinomial model (Bayesian SMM), a generative log-linear model in which a latent vector is drawn from an isotropic Gaussian prior and mapped through a softmax over a low-rank word subspace to multinomial word counts; variational inference returns a posterior mean and covariance for each document. The paper argues that the covariance encodes useful uncertainty, and it introduces a generative Gaussian linear classifier with uncertainty (GLCU) that uses that covariance when scoring classes. On 20Newsgroups and Fisher corpora, the authors report lower perplexity than the neural variational document model and topic-identification accuracy that beats unsupervised baselines and approaches a fully supervised CNN on 20Newsgroups.

What carries the argument

The load-bearing object is the variational posterior $q(w) = \mathcal{N}(w\,|\,\nu, \Gamma^{-1})$ over a document embedding, approximated with a diagonal-covariance Gaussian and optimized by maximizing the evidence lower bound. The intractable expectation over the softmax log-sum-exp term is handled with the reparameterization trick, writing $w = \nu + L\epsilon$ with $\epsilon \sim \mathcal{N}(0, I)$, so the offending term becomes a Monte Carlo average and the gradients stay low-variance. The shared model parameters are the background log-unigram vector $m$ and a low-rank total variability matrix $T$, whose rows are $\ell^1$-regularized with orthant-wise updates to give a sparse subspace. On top of these embeddings sits GLCU, a Gaussian linear classifier whose class-conditional likelihood for a document mean $\nu_d$ is $\mathcal{N}(\nu_d\,|\,\mu_\ell, \Gamma_d^{-1} + D^{-1})$, the sum of the model's shared within-class covariance and the document's inferred uncertainty; an EM algorithm estimates the class means and shared precision. This is the mechanism that turns uncertainty into a trained input to classification rather than a mere diagnostic.

What would settle it

Compute test perplexity for Bayesian SMM and NVDM using an estimator of true log-likelihood that does not depend on either model's variational bound, such as importance sampling with a very large number of samples, and check whether Bayesian SMM's advantage survives; if the gap shrinks or reverses, the reported advantage is an artifact of bound tightness rather than better data fit.

Watch

Extended reading notes

Core claim

The central claim is that Bayesian SMM's per-document posterior is not merely a by-product but the reason the model works: the covariance tells how much each embedding should be trusted, and using it in classification improves both accuracy and calibration. Concretely, on 20Newsgroups Bayesian SMM achieves corpus perplexities of 1043 and 882 for embedding dimensions 50 and 200, versus 1287 and 1387 for the neural variational document model (NVDM), and similar gaps under a 2000-word vocabulary; on Fisher manual transcriptions, Bayesian SMM with logistic regression reaches 89.91 percent topic-ID accuracy, while the TF-IDF baseline reaches 86.59 percent and a fine-tuned language-model baseline reaches 86.41 percent. On 20Newsgroups, Bayesian SMM with logistic regression reaches 84.65 percent, above all unsupervised baselines and close to the purely discriminative CNN's 86.12 percent. The authors also claim that GLCU, which uses the full posterior, produces lower cross-entropy than a Gaussian classifier that ignores uncertainty (0.68 versus 1.05 on Fisher manual transcriptions).

Load-bearing premise

The lower-perplexity claim assumes that each model's approximate score is a fair measure of how well the model fits the data, even though Bayesian SMM optimizes its score separately for every test document and NVDM uses a single shared encoder.

Editorial extensions

If this is right

  • Document embeddings extracted from a fully converged Bayesian SMM can be fed directly to linear classifiers for topic identification without early stopping, unlike SMM and NVDM embeddings.
  • Using the full posterior covariance in the GLCU classifier yields lower cross-entropy than using only means, so the uncertainty signal improves calibration, not just accuracy.
  • The reported perplexity results place Bayesian SMM above NVDM under both full and limited vocabularies on 20Newsgroups, and the model also fits Fisher speech data better than NVDM.
  • Unsupervised embeddings from Bayesian SMM plus a linear classifier match or beat supervised discriminative baselines on topic identification, despite receiving no topic labels during embedding training.
  • The same variational treatment extends to subspace n-gram models, allowing sentence and word embeddings to be learned with uncertainty.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper notes the posterior covariance's trace tracks document length; we infer that the covariance could serve as a confidence signal for retrieval, active learning, or rejection of low-quality embeddings, which the paper does not pursue.
  • A fair comparison that gives NVDM the same per-document ELBO optimization as Bayesian SMM would isolate whether the perplexity gap comes from the generative model or from bound tightness; the paper does not run this control.
  • GLCU's treatment of per-document covariance resembles scoring methods used in speaker recognition, and we infer that allowing class-specific rather than shared covariances in GLCU would be a natural next test on topic data.
  • Because Bayesian SMM is trained with per-document variational optimization rather than an amortized encoder, its robustness to overfitting may be a property of the inference scheme as much as the model family; applying the same scheme to NVDM's decoder would test this.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes Bayesian subspace multinomial model (Bayesian SMM), a generative log-linear bag-of-words model that represents each document as a Gaussian posterior over a low-dimensional embedding, thereby capturing uncertainty. Variational inference uses the reparameterization trick to handle the intractable log-sum-exp expectation, with ADAM updates and L1 regularization via orthant-wise learning. The authors also propose GLCU, a Gaussian linear classifier that incorporates posterior covariances, with an EM algorithm. Experiments on Fisher speech transcripts and 20Newsgroups report perplexity comparisons against NVDM and topic-identification comparisons against several baselines, including ULMFiT and a CNN. The paper claims state-of-the-art perplexity and strong or superior topic-ID accuracy, with robustness to overfitting and no need for early stopping.

Significance. If the results hold, the paper makes a useful contribution: uncertainty-aware document embeddings from a clean generative model, a transparent variational derivation, and a classifier that can exploit posterior covariances. The derivation of the ELBO gradients and the GLCU EM updates are laid out in detail and are consistent on inspection. The evaluation on held-out documents with cross-validation for hyperparameters is a strength, and the claimed robustness of Bayesian SMM over NVDM and SMM during training is interesting. However, the central perplexity claim is currently supported only by an uncontrolled comparison of variational upper bounds, so the significance of that claim is not established as written. The topic-ID results are more defensible, but some claims need qualification in the absence of significance testing.

major comments (2)
  1. [Section VII-B, Table II, Eqs. (41)-(42)] The perplexity comparison conflates model fit with bound tightness. Because log p(x) is intractable, the paper substitutes L(q), the ELBO, and correctly states that the resulting perplexities are upper bounds. However, for Bayesian SMM, q(w) for each test document is obtained by iteratively optimizing L(q) with ADAM (Section III-B), whereas for NVDM, q is obtained by a single forward pass through the trained encoder (Section V-E). Per-document optimization can only tighten the bound for Bayesian SMM, even if the true marginal likelihood is no better, so the large gaps in Table II (e.g., 851 vs 1519 at K=200) may be an artifact of bound tightness rather than of generative fit. The comparison is further confounded by the different number of MC samples used for the bound estimate (R=32 for Bayesian SMM versus R=20 for NVDM). To support the state-of-the-art perplexity claim, the two models should be compared with a matched estimator of log p(x), such as importance sampling or annealed importance sampling with the same inference budget and sample count, or the claim should be restricted to a statement about the optimized ELBO for a fixed inference procedure.
  2. [Section VII-B] The claim of 'state-of-the-art perplexity scores on 20Newsgroups' is not supported by the evidence in Table II, which compares Bayesian SMM only with NVDM. The paper relies on the prior result that NVDM beats LDA, docNADE, and DARN, but no numbers for those models are shown on the same splits, and other modern neural topic models are absent. To make the SOTA claim, the authors should either add direct baselines or qualify the claim to 'better than NVDM under the current evaluation setting.'
minor comments (5)
  1. [Abstract and Section VII-D] There are several language errors: 'the proposed model is outperforms state-of-the-art unsupervised topic models' and 'although the Bayesian is trained in an unsupervised fashion' should be corrected.
  2. [Equation (35) and Appendix B] The sign of the log-determinant term in Eq. (35) is inconsistent with Appendix B: the main text gives -D/2 log|D|, while the appendix derivation and the subsequent update in Eq. (37) are consistent with +D/2 log|D|. This should be fixed to avoid confusion.
  3. [Equation (22)] In the gradient for rows of T, the document word-count term is written as sum_k x_{ki}; this should presumably be the total word count of document d, i.e., sum_k x_{dk}. Please correct the indexing.
  4. [Section VI-B] The prior precision lambda and the number of samples R are free parameters, but the hyperparameter search is described only for K and omega. The fixed values used in the final experiments should be reported, since lambda directly affects posterior covariance and hence GLCU.
  5. [Tables III and IV] No statistical significance or variance estimates are reported. For example, the 20Newsgroups accuracy difference between Bayesian SMM+LR (84.65) and SCDV (84.60) is very small, so the statement that Bayesian SMM is 'better than all the other models except' the CNN should be qualified.

Circularity Check

0 steps flagged · score 0.0 of 10

No definitional circularity; perplexity-bound asymmetry is an evaluation concern, not a self-derived prediction.

full rationale

The paper's derivation chain is self-contained: Section II defines the generative model, Section III derives the ELBO and its gradients, Section IV derives the GLCU EM updates, and the experimental sections evaluate the learned embeddings on held-out test documents with topic labels withheld during embedding learning. The self-citations to prior work by the same group, e.g. [20] for the l1 regularization of the subspace multinomial model and [30] for covariance-aware scoring, are not load-bearing for the central claims; they support implementation choices, not the main result. The perplexity comparison in Section VII-B is a potential fairness issue rather than a circular reduction: the paper acknowledges that true log p(x) is intractable and that only the lower bound L(q) can be computed, giving upper-bound perplexity values. The fact that Bayesian SMM optimizes q per test document while NVDM obtains q from an encoder makes the bounds differ in tightness, but the reported quantity is still an acknowledged variational approximation to the same model-evaluation objective, not a renamed fitting target. The topic-identification claims in Tables III and IV are supervised evaluations on held-out labels and are not constructed from the model definition. No step has been found where a prediction is equivalent to its input by construction or where a load-bearing premise rests solely on a self-citation chain.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The model rests on standard variational inference plus domain assumptions about how text is generated. No ad hoc entities are introduced. The free parameters are all hyperparameters selected by hand or cross-validation; none are fitted to the held-out evaluation metric.

free parameters (4)
  • Prior precision lambda = 10 in main experiments (N(0, diag(0.1)) for both prior and variational initialization)
    Controls the strength of the Gaussian prior over document embeddings; chosen by hand and not learned, and it affects all posterior estimates.
  • Embedding dimension K = 50, 100, 200, up to 800 across experiments
    Selected from {100,...,800} for topic ID and set to 50 or 200 for perplexity; a standard capacity hyperparameter.
  • L1 regularization weight omega = Search range {0.0001,...,10.0}
    Controls sparsity of the word subspace matrix T; tuned via cross-validation for topic ID and affects the learned embeddings.
  • Number of Monte Carlo samples R = 32 for perplexity; unspecified during training
    Used in the reparameterized estimate of the log-sum-exp expectation; affects gradient variance and the ELBO estimate.
assumptions (6)
  • domain assumption Documents are independent and each is generated from a latent embedding w drawn from an isotropic Gaussian prior.
    Eq. (1) and Fig. 1; all derivations treat documents independently, which ignores inter-document structure.
  • domain assumption Word counts follow a Multinomial distribution whose logits are a linear function of the latent embedding: x ~ Multi(softmax(m + T w)).
    Eqs. (2)-(3); this is the likelihood model that ties embeddings to observed bags of words.
  • domain assumption The variational posterior is restricted to a diagonal Gaussian family q(w) = N(nu, diag(exp{2 sigma})).
    Eqs. (18) and (43); the paper calls this a simplification, but it limits uncertainty estimates to per-dimension variances.
  • domain assumption The ELBO is a valid proxy for marginal log-likelihood when computing perplexity.
    Section VII-B states log p(x) is intractable and uses L(q) from Eq. (16), so reported perplexities are upper bounds on true values.
  • standard math Variational inference with the reparameterization trick gives unbiased low-variance gradient estimates for the log-sum-exp expectation.
    Section III, Eqs. (14)-(15); standard result from Kingma and Welling (2014), cited as [14].
  • domain assumption For GLCU, each class is Gaussian with a shared precision matrix and the document posterior covariance is additive noise.
    Section IV, Eqs. (29)-(30); this is an assumption about the generative process of class labels, not derived from the text model.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Learning document embeddings along with their uncertainties." pith.science (2026). https://pith.science/paper/VGBHSI6Z

@misc{pith2026190807599,
  author       = {Pith},
  title        = {Pith review of: Learning document embeddings along with their uncertainties},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VGBHSI6Z}},
  note         = {Machine review of arXiv:1908.07599}
}
read the original abstract

Majority of the text modelling techniques yield only point-estimates of document embeddings and lack in capturing the uncertainty of the estimates. These uncertainties give a notion of how well the embeddings represent a document. We present Bayesian subspace multinomial model (Bayesian SMM), a generative log-linear model that learns to represent documents in the form of Gaussian distributions, thereby encoding the uncertainty in its co-variance. Additionally, in the proposed Bayesian SMM, we address a commonly encountered problem of intractability that appears during variational inference in mixed-logit models. We also present a generative Gaussian linear classifier for topic identification that exploits the uncertainty in document embeddings. Our intrinsic evaluation using perplexity measure shows that the proposed Bayesian SMM fits the data better as compared to the state-of-the-art neural variational document model on Fisher speech and 20Newsgroups text corpora. Our topic identification experiments show that the proposed systems are robust to over-fitting on unseen test data. The topic ID results show that the proposed model is outperforms state-of-the-art unsupervised topic models and achieve comparable results to the state-of-the-art fully supervised discriminative models.

Figures

Figures reproduced from arXiv: 1908.07599 by the authors.

Figure 1
Figure 1. Graphical model for Bayesian SMM p(w | xb). Parameters of such posterior distribution can be then used as a low dimensional representation of the document. Note that such distribution also encodes the inferred uncer￾tainty about such representation. Using Bayes’ rule, the posterior distribution of document embedding w is written as1 : p(w|x) = p(x|w)p(w) R p(x|w)p(w) dw . (4) In numerator of (4), p(w) represents pri… view at source ↗
Figure 1
Figure 1. Hence, we derive the inference only for one document [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Graphical model for LDA B. Classification Given a test document embedding posterior distribution q(w) = N (w | ν,Γ −1 ), we compute the class conditional likelihood according to (30), and the posterior probability of a class Ck is obtained by applying the Bayes’ rule: p(Ck | ν,Γ, Θ) = p(ν | µk, D,Γ) p(Ck) P ` p(ν | µ`, D,Γ) p(C`) . (38) V. RELATED MODELS In this section, we review and relate some of the popular PTMs… view at source ↗
Figures from the paper (5 more)
Figure 3
Figure 3. Figure 3: Graphical model for CTM B. Correlated topic model The generative process for a document in CTM [22] is same as in LDA, except for document vectors are now drawn from Gaussian, i.e., p(η) = N (η | µ, diag(λ) −1 ), (39) θ = softmax(η). (40) In this formulation, the docum…
Figure 4
Figure 4. Figure 4: Convergence of Bayesian SMM for various initial [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Comparison of training and test data perplexities obtained using Bayesian SMM and NVDM for both [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Performance of topic ID systems on Fisher data at various checkpoints during model training. The circular dot (•) represents the best cross-validation score and the corresponding test score obtained using the early stopping mechanism (ESM). The embedding dimension was …
Figure 7
Figure 7. Figure 7: Uncertainty (trace of covariance of posterior distribu [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 45 canonical work pages

  1. [1]

    LDA-based document models for ad-hoc retrieval,

    X. Wei and W. B. Croft, “LDA-based document models for ad-hoc retrieval,” inProc. of the 29th Annual International ACM SIGIR, August 2006, pp. 178–185

    work page 2006

  2. [2]

    Context dependent recurrent neural network language model,

    T. Mikolov and G. Zweig, “Context dependent recurrent neural network language model,” in IEEE SLT Workshop, December 2012, pp. 234–239

    work page 2012

  3. [3]

    Limited resource term detection for effective topic identification of speech,

    J. Wintrode and S. Khudanpur, “Limited resource term detection for effective topic identification of speech,” in IEEE ICASSP , May 2014, pp. 7118–7122

    work page 2014

  4. [4]

    Recurrent neural network language model adaptation for multi-genre broadcast speech recognition,

    X. Chen, T. Tan, X. Liu, P. Lanchantin, M. Wan, M. J. F. Gales, and P. C. Woodland, “Recurrent neural network language model adaptation for multi-genre broadcast speech recognition,” in Proc. Interspeech. ISCA, September 2015, pp. 3511–3515

    work page 2015

  5. [5]

    i-Vectors in Language Modeling: An Efficient Way of Domain Adaptation for Feed-Forward Models,

    K. Bene ˇs, S. Kesiraju, and L. Burget, “i-Vectors in Language Modeling: An Efficient Way of Domain Adaptation for Feed-Forward Models,” in Proc. Interspeech. ISCA, 2018, pp. 3383–3387

    work page 2018

  6. [6]

    Distributed representations of words and phrases and their composi- tionality,

    T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean, “Distributed representations of words and phrases and their composi- tionality,” in Advances in NIPS , December 2013, pp. 3111–3119

    work page 2013

  7. [7]

    GloVe: Global Vectors for Word Representation,

    J. Pennington, R. Socher, and C. D. Manning, “GloVe: Global Vectors for Word Representation,” in Proc. of the 2014 Conference on EMNLP, ACL, October 2014, pp. 1532–1543

    work page 2014

  8. [8]

    Distributed representations of sentences and documents,

    Q. V . Le and T. Mikolov, “Distributed representations of sentences and documents,” in Proc. of the ICML , June 2014, pp. 1188–1196

    work page 2014

Show all 46 references
  1. [9]

    Universal Language Model Fine-tuning for Text Classification,

    J. Howard and S. Ruder, “Universal Language Model Fine-tuning for Text Classification,” in Proc. of the 56th Annual Meeting of the ACL . Melbourne, Australia: ACL, Jul. 2018, pp. 328–339

    2018

  2. [10]

    Deep contextualized word representations,

    M. Peters, M. Neumann, M. Iyyer, M. Gardner, C. Clark, K. Lee, and L. Zettlemoyer, “Deep contextualized word representations,” in Proc. of the NAACL: HLT. ACL, Jun. 2018, pp. 2227–2237

    2018

  3. [11]

    BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,

    J. Devlin, M. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding,” CoRR, vol. abs/1810.04805v1, 2018

    2018 arXiv

  4. [12]

    Latent variable models,

    C. Bishop, “Latent variable models,” in Learning in Graphical Models . MIT Press, January 1999, pp. 371–403

    1999

  5. [13]

    Probabilistic topic models,

    D. M. Blei, “Probabilistic topic models,” Commun. ACM, vol. 55, no. 4, pp. 77–84, Apr. 2012

    2012

  6. [14]

    Auto-Encoding Variational Bayes,

    D. P. Kingma and M. Welling, “Auto-Encoding Variational Bayes,” in Proc. of the 2nd ICLR , 2014

    2014

  7. [15]

    Neural variational inference for text processing,

    Y . Miao, L. Yu, and P. Blunsom, “Neural variational inference for text processing,” in Proceedings of the 33rd ICML , ser. ICML’16. JMLR.org, 2016, pp. 1727–1736

    2016

  8. [16]

    C. M. Bishop, Pattern Recognition and Machine Learning (Information Science and Statistics). Secaucus, NJ, USA: Springer-Verlag New York, Inc., 2006

    2006

  9. [17]

    Stochastic backpropa- gation and approximate inference in deep generative models,

    D. J. Rezende, S. Mohamed, and D. Wierstra, “Stochastic backpropa- gation and approximate inference in deep generative models,” in Proc. of the 31st ICML , ser. Proc. of Machine Learning Research, E. P. Xing and T. Jebara, Eds., vol. 32. Bejing, China: PMLR, 22–24 Jun 2014, pp...

    2014

  10. [18]

    Reg- ularized subspace n-gram model for phonotactic ivector extraction,

    M. Soufifar, L. Burget, O. Plchot, S. Cumani, and J. Cernock ´y, “Reg- ularized subspace n-gram model for phonotactic ivector extraction,” in INTERSPEECH. ISCA, Aug 2013, pp. 74–78

    2013

  11. [19]

    Topic identification and discovery on text and speech,

    C. May, F. Ferraro, A. McCree, J. Wintrode, D. Garcia-Romero, and B. V . Durme, “Topic identification and discovery on text and speech,” in Proc. of the 2015 Conference on EMNLP, September 2015, pp. 2377– 2387

    2015

  12. [20]

    Learning Document Representations Using Subspace Multinomial Model,

    S. Kesiraju, L. Burget, I. Sz ¨oke, and J. ˇCernock´y, “Learning Document Representations Using Subspace Multinomial Model,” in Proc. of IN- TERSPEECH. ISCA, September 2016, pp. 700–704

    2016

  13. [21]

    Latent Dirichlet Allocation,

    D. M. Blei, A. Y . Ng, and M. I. Jordan, “Latent Dirichlet Allocation,” JMLR, vol. 3, pp. 993–1022, 2003

    2003

  14. [22]

    Correlated topic models,

    D. M. Blei and J. D. Lafferty, “Correlated topic models,” in Advances in Neural Information Processing Systems NIPS , December 2005, pp. 147–154

    2005

  15. [23]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” in 3rd ICLR, May 2015

    2015

  16. [24]

    Scalable Training of L1-Regularized Log-Linear Models,

    G. Andrew and J. Gao, “Scalable Training of L1-Regularized Log-Linear Models,” in Proc. of the 24th ICML . New York, USA: ACM, 2007, pp. 33–40

    2007

  17. [25]

    Prosodic speaker verification using subspace multinomial models with intersession compensation,

    M. Kockmann, L. Burget, O. Glembek, L. Ferrer, and J. ˇCernock´y, “Prosodic speaker verification using subspace multinomial models with intersession compensation,” inProc. of INTERSPEECH. ISCA, Septem- ber 2010, pp. 1061–1064

    2010

  18. [26]

    Front- end factor analysis for speaker verification,

    N. Dehak, P. Kenny, R. Dehak, P. Dumouchel, and P. Ouellet, “Front- end factor analysis for speaker verification,” IEEE Trans. Audio, Speech & Language Processing , vol. 19, no. 4, pp. 788–798, 2011

    2011

  19. [27]

    A comparison of variational ap- proximations for fast inference in mixed logit models,

    N. Depraetere and M. Vandebroek, “A comparison of variational ap- proximations for fast inference in mixed logit models,” Computational Statistics, vol. 32, no. 1, pp. 93–125, 2017

    2017

  20. [28]

    The Matrix Cookbook,

    K. B. Petersen and M. S. Pedersen, “The Matrix Cookbook,” Nov 2012

    2012

  21. [29]

    PLDA for speaker verification with utterances of arbitrary duration,

    P. Kenny, T. Stafylakis, P. Ouellet, M. J. Alam, and P. Dumouchel, “PLDA for speaker verification with utterances of arbitrary duration,” in 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, May 2013, pp. 7649–7653

    2013

  22. [30]

    Exploiting i-vector posterior co- variances for short-duration language recognition,

    S. Cumani, O. Plchot, and R. F ´er, “Exploiting i-vector posterior co- variances for short-duration language recognition,” in Proc. of INTER- SPEECH, no. 09. ISCA, 2015, pp. 1002–1006

    2015

  23. [31]

    iVector Approach to Phonotactic Language Recognition,

    M. Soufifar, M. Kockmann, L. Burget et al. , “iVector Approach to Phonotactic Language Recognition,” inProc. of INTERSPEECH. ISCA, August 2011, pp. 2913–2916

    2011

  24. [32]

    Learning word vectors for sentiment analysis,

    A. L. Maas, R. E. Daly, P. T. Pham, D. Huang, A. Y . Ng, and C. Potts, “Learning word vectors for sentiment analysis,” in The 49th Annual Meeting of the ACL: Human Language Technologies , June 2011, pp. 142–150

    2011

  25. [33]

    Sparse Additive Generative Models of Text,

    J. Eisenstein, A. Ahmed, and E. P. Xing, “Sparse Additive Generative Models of Text,” in Proc. of the 28th ICML . USA: Omnipress, 2011, pp. 1041–1048

    2011

  26. [34]

    Sparse Overcomplete Latent Variable Decomposition of Counts Data,

    M. V . S. Shashanka, B. Raj, and P. Smaragdis, “Sparse Overcomplete Latent Variable Decomposition of Counts Data,” in NIPS, December 2007, pp. 1313–1320

    2007

  27. [35]

    Sparse Topical Coding,

    J. Zhu and E. P. Xing, “Sparse Topical Coding,” in Proc. of the 27th Conference on UAI, July 2011, pp. 831–838

    2011

  28. [36]

    Scdv : Sparse composite document vectors using soft clustering over distributional rep- resentations,

    D. Mekala, V . Gupta, B. Paranjape, and H. Karnick, “Scdv : Sparse composite document vectors using soft clustering over distributional rep- resentations,” inProc. of the 2017 Conference on EMNLP. Copenhagen, Denmark: ACL, Sep. 2017, pp. 659–669

    2017

  29. [37]

    Topic Identification from Audio Recordings using Word and Phone Recognition Lattices,

    T. J. Hazen, F. Richardson, and A. Margolis, “Topic Identification from Audio Recordings using Word and Phone Recognition Lattices,” inIEEE Workshop on ASRU, December 2007, pp. 659–664

    2007

  30. [38]

    MCE Training Techniques for Topic Identification of Spoken Audio Documents,

    T. J. Hazen, “MCE Training Techniques for Topic Identification of Spoken Audio Documents,” IEEE Transactions on Audio, Speech, and Language Processing, vol. 19, no. 8, pp. 2451–2460, Nov 2011

    2011

  31. [39]

    The Kaldi Speech Recognition Toolkit,

    D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y . Qian, P. Schwarz, J. Silovsky, G. Stem- mer, and K. Vesely, “The Kaldi Speech Recognition Toolkit,” in IEEE Workshop on ASRU. IEEE Signal Processing Society, Dec 2011

    2011

  32. [40]

    Sequence- discriminative training of deep neural networks,

    K. Vesel ´y, A. Ghoshal, L. Burget, and D. Povey, “Sequence- discriminative training of deep neural networks,” in Proc. of INTER- SPEECH. ISCA, August 2013, pp. 2345–2349

    2013

  33. [41]

    Modeling documents with a deep boltzmann machine,

    N. Srivastava, R. Salakhutdinov, and G. Hinton, “Modeling documents with a deep boltzmann machine,” in Proc. of the Twenty-Ninth Confer- ence on UAI , ser. UAI’13. Arlington, Virginia, United States: AUAI Press, 2013, pp. 616–624

    2013

  34. [42]

    A neural autoregressive topic model,

    H. Larochelle and S. Lauly, “A neural autoregressive topic model,” in Advances in NIPS , December 2012, pp. 2717–2725

    2012

  35. [43]

    Neural variational inference and learning in belief networks,

    A. Mnih and K. Gregor, “Neural variational inference and learning in belief networks,” in Proc. of the 31th ICML , June 2014, pp. 1791–1799

    2014

  36. [44]

    Joint verification-identification in end-to-end multi-scale cnn framework for topic identification,

    R. Pappagari, J. Villalba, and N. Dehak, “Joint verification-identification in end-to-end multi-scale cnn framework for topic identification,” in IEEE ICASSP, April 2018, pp. 6199–6203

    2018

  37. [45]

    Learning context-sensitive word embed- dings with neural tensor skip-gram model,

    P. Liu, X. Qiu, and X. Huang, “Learning context-sensitive word embed- dings with neural tensor skip-gram model,” in Proc. of the 24th Inter- national Conference on Artificial Intelligence , ser. IJCAI’15. AAAI Press, 2015, pp. 1284–1290

    2015

  38. [46]

    Gaussian meta- embeddings for efficient scoring of a heavy-tailed PLDA model,

    N. Br ¨ummer, A. Silnova, L. Burget, and T. Stafylakis, “Gaussian meta- embeddings for efficient scoring of a heavy-tailed PLDA model,” in Proc. Odyssey 2018 The Speaker and Language Recognition Workshop , 2018, pp. 349–356

    2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.