REVIEW 13 cited by
Language-agnostic BERT Sentence Embedding
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
While BERT is an effective method for learning monolingual sentence embeddings for semantic similarity and embedding based transfer learning (Reimers and Gurevych, 2019), BERT based cross-lingual sentence embeddings have yet to be explored. We systematically investigate methods for learning multilingual sentence embeddings by combining the best methods for learning monolingual and cross-lingual representations including: masked language modeling (MLM), translation language modeling (TLM) (Conneau and Lample, 2019), dual encoder translation ranking (Guo et al., 2018), and additive margin softmax (Yang et al., 2019a). We show that introducing a pre-trained multilingual language model dramatically reduces the amount of parallel training data required to achieve good performance by 80%. Composing the best of these methods produces a model that achieves 83.7% bi-text retrieval accuracy over 112 languages on Tatoeba, well above the 65.5% achieved by Artetxe and Schwenk (2019b), while still performing competitively on monolingual transfer learning benchmarks (Conneau and Kiela, 2018). Parallel data mined from CommonCrawl using our best model is shown to train competitive NMT models for en-zh and en-de. We publicly release our best multilingual sentence embedding model for 109+ languages at https://tfhub.dev/google/LaBSE.
Forward citations
Cited by 13 Pith papers
-
Bekko Embedding: Parameter-Efficient Multilingual Retrieval with Ultra-Compact Encoders
Bekko a8m, with 7.7M active parameters, scores 56.2 on MMTEB Multilingual v2 Retrieval, beating mE5 models and BGE-M3, while a25m reaches 57.5, on par with gte-multilingual-base.
-
Moral Semantics Survive Machine Translation: Cross-Lingual Evidence from Moral Foundations Corpora
Machine translation preserves moral semantics in Polish social media data well enough for cross-lingual use, shown by 0.86 mean embedding similarity and 0.01-0.02 AUC gaps in moral foundations classification.
-
Intertextual Parallel Detection in Biblical Hebrew: A Transformer-Based Benchmark
E5 and AlephBERT embeddings rank known Samuel/Kings-Chronicles parallels above non-parallel verses, but the benchmark restricts searches to Samuel/Kings and lacks baselines.
-
Align-then-Slide: A complete evaluation framework for Ultra-Long Document-Level Machine Translation
An evaluation framework for document-level MT that aligns source and target sentences via dynamic programming and averages 1-4 chunk sliding-window scores, reporting 0.929 Pearson correlation with human MQM and enabli...
-
VIBE: Video-Input Brain Encoder for fMRI Response Modeling
A two-stage multimodal Transformer with fixed pretrained feature extractors predicts fMRI activity from movies with Pearson r = 0.32 (in-domain) and 0.21 (out-of-domain), beating the challenge baseline by about 0.12.
-
AI or Human? Understanding Perceptions of Embodied Robots with LLMs
In an embodied Turing Test with a physical robot, 34 participants could not identify AI versus human control above chance, and human operators were misidentified as AI far more often than the reverse.
-
The first open machine translation system for the Chechen language
A 171K-pair Chechen-Russian parallel corpus plus a fine-tuned NLLB-200 model are released, giving the first open Chechen-Russian translation system with human-evaluated quality near Google Translate.
-
ConECT Dataset: Overcoming Data Scarcity in Context-Aware E-Commerce MT
A new Czech-to-Polish e-commerce translation dataset is released, and the paper shows small improvements from visual and category context, with a negative result for image descriptions.
-
Building a Few-Shot Cross-Domain Multilingual NLU Model for Customer Care
The proposed teacher-student distillation with isotropic regularization improves few-shot multilingual intent detection on proprietary customer care data, though the advertised gains are against baselines not given th...
-
SwitchLingua: The First Large-Scale Multilingual and Multi-Ethnic Code-Switching Dataset
The authors present SwitchLingua, a large multilingual code-switching text and audio dataset, and SAER, a semantic-aware error metric for code-switching ASR evaluation.
-
GATE: General Arabic Text Embedding for Enhanced Semantic Textual Similarity with Matryoshka Representation Learning and Hybrid Loss Training
GATE's Arabic-Triplet-Matryoshka-V2 reports the highest average scores on the MTEB Arabic STS17/STS22/STS22-v2 tasks among the models compared in the paper.
-
Hard Negative Mining for Domain-Specific Retrieval in Enterprise Systems
A reranker fine-tuned on hard negatives selected by two cosine-distance criteria outperforms older negative sampling methods on enterprise and domain-specific retrieval benchmarks.
-
Intent Classification on Low-Resource Languages with Query Similarity Search
A k-nearest-neighbor search over multilingual query embeddings provides zero-shot intent classification for low-resource languages, with accuracy below translation-based and supervised baselines.
Discussion (0). Sign in to comment.