Pith. sign in

REVIEW 18 cited by

Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2004.09813 v2 pith:HDDMEYW6 submitted 2020-04-21 cs.CL

Nils Reimers , Iryna Gurevych This is my paper
classification cs.CL
keywords sentenceembeddingslanguagesmodelsextendmonolingualmultilingualoriginal
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We present an easy and efficient method to extend existing sentence embedding models to new languages. This allows to create multilingual versions from previously monolingual models. The training is based on the idea that a translated sentence should be mapped to the same location in the vector space as the original sentence. We use the original (monolingual) model to generate sentence embeddings for the source language and then train a new system on translated sentences to mimic the original model. Compared to other methods for training multilingual sentence embeddings, this approach has several advantages: It is easy to extend existing models with relatively few samples to new languages, it is easier to ensure desired properties for the vector space, and the hardware requirements for training is lower. We demonstrate the effectiveness of our approach for 50+ languages from various language families. Code to extend sentence embeddings models to more than 400 languages is publicly available.

Discussion (0). Sign in to comment.

Forward citations

Cited by 18 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Cordyceps: Covert Control Attacks on LLMs via Data Poisoning

    cs.CR 2026-05 unverdicted novelty 8.0 of 10

    Cordyceps poisoning induces an information hiding scheme in LLMs via semantic associations, enabling covert control attacks with 40% higher success than prior methods and up to 98% survival against defenses.

  2. Bekko Embedding: Parameter-Efficient Multilingual Retrieval with Ultra-Compact Encoders

    cs.IR 2026-07 conditional novelty 7.0 of 10

    Bekko a8m, with 7.7M active parameters, scores 56.2 on MMTEB Multilingual v2 Retrieval, beating mE5 models and BGE-M3, while a25m reaches 57.5, on par with gte-multilingual-base.

  3. The Harder Text Embedding Benchmark (HTEB): Beyond One-dimensional Static Robustness

    cs.CL 2026-05 unverdicted novelty 7.0 of 10

    HTEB introduces dynamic, multi-axis evaluation of text embedding robustness using LLM transformations, finding decoupled profiles across models and that scaling does not close all robustness gaps.

  4. Identifying and Characterizing Semantic Clones of Solidity Functions

    cs.SE 2026-04 unverdicted novelty 7.0 of 10

    A code-and-comment analysis method detects semantic clones in Solidity functions with 59% overall precision (84% for same-name functions) and 97% recall on 300k contracts, plus LLM summaries for uncommented code.

  5. A Good Talk Does not Look Like a Summary, It Teaches You! Measuring Takeaways from Paper-to-Video Talks

    cs.MM 2026-06 unverdicted novelty 6.0 of 10

    EffectivePresentationScorer evaluates paper-to-video talks for instructional quality by checking clear explanation of ideas, prerequisite concepts, and links to contributions, finding that current systems cover topics...

  6. Generalizing Graph Foundation Models via Hyperbolic Retrieval-Augmented Generation

    cs.IR 2026-06 unverdicted novelty 6.0 of 10

    HyRAG improves zero-shot generalization of graph foundation models by indexing and retrieving from tree-structured knowledge in hyperbolic space via multi-granularity retrieval and dual-path fusion.

  7. SeedER: Seed-and-Expand Retrieval from Knowledge Graphs

    cs.LG 2026-05 unverdicted novelty 6.0 of 10

    SeedER uses initial dense seeding followed by RL-driven selective expansion to improve recall on compositional KG queries while limiting candidate set size.

  8. LLaVA-Video: Video Instruction Tuning With Synthetic Data

    cs.CV 2024-10 unverdicted novelty 6.0 of 10

    LLaVA-Video-178K is a new synthetic video instruction dataset that, when combined with existing data to train LLaVA-Video, produces strong results on video understanding benchmarks.

  9. Adapting Multilingual Embedding Models to Turkish via Cross-Lingual Tokenizer Surgery and Offline Distillation

    cs.CL 2026-05 unverdicted novelty 5.0 of 10

    A 200M-parameter Turkish sentence embedding model is adapted from a multilingual teacher via tokenizer pruning, mean-composition initialization, and offline cosine distillation, achieving 77.55% Pearson correlation on...

  10. Towards Better Static Code Analysis Reports: Sentence Transformer-based Filtering of Non-Actionable Alerts

    cs.SE 2026-04 conditional novelty 5.0 of 10

    STAF applies sentence embeddings from transformers to classify SCA findings, reaching 89% F1 and beating prior filters by 11% within projects and 6% across projects.

  11. DeMeVa at LeWiDi-2025: Modeling Perspectives with In-Context Learning and Label Distribution Learning

    cs.CL 2025-09 conditional novelty 5.0 of 10

    In-context learning with up to 10 demonstrations per annotator can mimic individual annotator behavior well enough to produce competitive soft labels on four disagreement-rich datasets.

  12. Leveraging Large Language Models for Accurate Sign Language Translation in Low-Resource Scenarios

    cs.CL 2025-08 unverdicted novelty 5.0 of 10

    A prompting method that links signs to short text descriptions lets large language models translate English and Italian into sign language glosses, beating prior models in low-data settings.

  13. CCL-XCoT: An Efficient Cross-Lingual Knowledge Transfer Method for Mitigating Hallucination Generation

    cs.CL 2025-07 conditional novelty 5.0 of 10

    CCL-XCoT combines curriculum-based contrastive pretraining with cross-lingual chain-of-thought fine-tuning, lifting hallucination-free rates in low-resource QA from 1-18% to 55-74%.

  14. Preserving Knowledge in Large Language Model with Model-Agnostic Self-Decompression

    cs.CL 2024-06 unverdicted novelty 5.0 of 10

    Introduces Tree Generation (TG-SFT) to generate synthetic instruction-tuning data from LLMs, reducing catastrophic forgetting when fine-tuning MLLMs on domain-specific or multimodal data.

  15. Private, Verifiable, and Auditable AI Systems

    cs.CR 2025-08 conditional novelty 4.0 of 10

    A thesis demonstrating partial prototypes for zk-verifiable model evaluation and privacy-preserving retrieval, and arguing these pieces can compose into end-to-end auditable AI systems.

  16. AdaDocVQA: Adaptive Framework for Long Document Visual Question Answering in Low-Resource Settings

    cs.CL 2025-08 conditional novelty 4.0 of 10

    A hybrid retrieval, data augmentation, and ensemble inference framework fine-tunes Qwen2.5-VL-72B to 59% on the Japanese LAVA document VQA benchmark.

  17. Never Come Up Empty: Adaptive HyDE Retrieval for Improving LLM Developer Support

    cs.SE 2025-07 conditional novelty 4.0 of 10

    A HyDE retrieval pipeline with full-answer context and adaptive similarity thresholding improves LLM answers to Stack Overflow questions over zero-shot prompting for three of four open-source models.

  18. FullRecall: A Semantic Search-Based Ranking Approach for Maximizing Recall in Patent Retrieval

    cs.IR 2025-07 reject novelty 4.0 of 10

    A three-phase patent retrieval pipeline achieved 100% recall on five examiner-cited test queries, but the score is driven by post hoc cutoff choices and a candidate set that already contains the target patents.

Pith tools