Pith. sign in

REVIEW 5 cited by

Probabilistic FastText for Multi-Sense Word Embeddings

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1806.02901 v1 pith:26NKQVLB submitted 2018-06-07 cs.CL cs.AIcs.LGstat.ML

Ben Athiwaratkun , Andrew Gordon Wilson , Anima Anandkumar This is my paper · ORCID
classification cs.CLcs.AIcs.LGstat.ML
keywords probabilisticwordfasttextmodelembeddingsmixtureachievebenchmarks
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

We introduce Probabilistic FastText, a new model for word embeddings that can capture multiple word senses, sub-word structure, and uncertainty information. In particular, we represent each word with a Gaussian mixture density, where the mean of a mixture component is given by the sum of n-grams. This representation allows the model to share statistical strength across sub-word structures (e.g. Latin roots), producing accurate representations of rare, misspelt, or even unseen words. Moreover, each component of the mixture can capture a different word sense. Probabilistic FastText outperforms both FastText, which has no probabilistic model, and dictionary-level probabilistic embeddings, which do not incorporate subword structures, on several word-similarity benchmarks, including English RareWord and foreign language datasets. We also achieve state-of-art performance on benchmarks that measure ability to discern different meanings. Thus, the proposed model is the first to achieve multi-sense representations while having enriched semantics on rare words.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. NushuRescue: Revitalization of the Endangered Nushu Language with AI

    cs.CL 2024-11 conditional novelty 6.0 of 10

    A 35-example few-shot GPT-4-Turbo pipeline reached 48.69% exact-match translation accuracy on held-out Nushu sentences and produced a 98-sentence silver corpus, alongside the first public Nushu-Chinese dataset.

  2. Modeling Islamist Extremist Communications on Social Media using Contextual Dimensions: Religion, Ideology, and Hate

    cs.SI 2019-08 reject novelty 6.0 of 10

    A tri-dimensional religion-ideology-hate embedding model reaches 0.97 precision for identifying Islamist extremist Twitter users, a 10.2% relative gain over a re-implemented baseline.

  3. SemGes: Semantics-aware Co-Speech Gesture Generation using Semantic Coherence and Relevance Learning

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A two-stage VQ-VAE and crossmodal transformer with coherence and relevance losses produces semantically aware co-speech gestures, beating four baselines on BEAT and TED Expressive for FGD, diversity, and SRGR.

  4. Low-Resource Neural Machine Translation Using Recurrent Neural Networks and Transfer Learning: A Case Study on English-to-Igbo

    cs.CL 2025-04 reject novelty 3.0 of 10

    Applying known RNN and transfer-learning methods to English-Igbo yields modest BLEU scores, but the claimed +4.83 BLEU improvement over baselines is inconsistent with the paper's own tables.

  5. A Multi-tiered Solution for Personalized Baggage Item Recommendations using FastText and Association Rule Mining

    cs.IR 2025-01 reject novelty 3.0 of 10

    A four-phase baggage recommendation system using FastText similarity and Apriori association rules is described, but its effectiveness claims are not supported by an independent evaluation.

Pith tools